The reason why the vowels in the first syllables of 'south' and 'southern' are different is Trisyllabic Laxing (TSL) which was a phonological rule back in Old and Middle English. It's a process whereby a long vowel/diphthong is shortened if two or more syllables follow.
'Southern' was a three-syllable word when TSL applied to it. TSL shortened the vowel in its first syllable, therefore it has a short vowel even in Modern English. There are some other pairs showing the same distinction such as derive-derivative, divine-divinity, pronounce-pronunciation, serene-serenity, impede-impediment, insane-insanity, profane-profanity etc.
It's a process whereby a tense vowel (long vowel or a diphthong) is laxed (shortened) if two (or more) syllables follow.
As we add syllables to the base of a word in English, we tend to reduce the length of the vowel in the base. If a syllable having tense vowel is followed by two or more syllables, the tense vowel often becomes lax.
At one point, this rule applied to all relevant cases; it was therefore purely a phonological rule, a constraint upon what was pronounceable in English. Later on, it ceased to be a part of English phonology, however, its remnants are still highly visible in Modern English.
- insane /ɪnˈseɪn/ → insanity /ɪnˈsæn.ə.ti/
- sincere /sɪnˈsɪə/ → sincerity /sɪnˈsɛ.rə.ti/
- serene /səˈriːn/ → serenity /səˈrɛ.nə.ti/
- divine /dɪˈvaɪn/ → divinity /dɪˈvɪ.nə.ti/
- pronounce /prəˈnaʊns/ → pronunciation /prəˌnʌn.siˈeɪ.ʃ(ə)n/
- provoke /prəˈvəʊk/ → provocative /prəˈvɒk.ə.tɪv/
Another interesting example is holy - holiday.
- Holy /ˈhəʊli/ → holiday /ˈhɒlɪdeɪ/
As you can see, there are at least two syllables after the tense vowels, so they get laxed because of TSL.
There's a fairly regular pattern of the vowel change in the words I mentioned above. It can be summarised as:
- /eɪ/ → /æ/
- /ɪə/ → /ɛ/
- /iː/ → /ɛ/
- /aɪ/ → /ɪ/
- /aʊ/ → /ʌ/
- /əʊ/ → /ɒ/ (AmE: /ɑ/)
In Modern English, there are systematic exceptions to the process, such as in words ending in -ness (e.g. mindfulness, loneliness etc). There are also occasional, non-systematic exceptions including later borrowings such as obese-obesity.
- Obese /əʊˈbiːs/ → obesity /əʊˈbiː.sə.ti/ not /əʊˈbɛ.sə.ti/
- Nightingale /ˈnaɪ.tɪŋ.ɡeɪl/
- Some words having the suffix -able such as signable /saɪnəb(ə)l/, boatable /ˈbəʊtəb(ə)l/ etc.
There are lots of other exceptions, however.
The above changes happen in trisyllabic (three-syllable) or polysyllabic words, but 'southern' is not even a three-syllable word, why then is it pronounced with a short vowel?
The short vowel in 'southern'
The history of southern is more interesting. Wikipedia also mentions disyllabic laxing (when one syllable follows the tense vowel as in 'southern'), though the vowel in 'southern' is not the result of that. It is because of trisyllabic laxing.
How come Trisyllabic laxing caused 'southern' to have a short vowel instead of a long vowel?
The anomaly here is caused by historical sound changes. 'Southern' used to be a three-syllable word when Trisyllabic Laxing applied.
'South' was sūþ (or sūð)1 in Old English and it was pronounced /suːθ/. While 'southern' was sūþerne2 (or sūðerne) pronounced /ˈsuː.θer.ne/ ([ˈsuː.ðer.ne])
In Middle English, 'southern' was pronounced /ˈsuː.ðər.nə/. Then Trisyllabic Laxing applied to it and reduced the long vowel /uː/ to /ʊ/ (the vowel in put).
Then in Early Modern English, there was another sound change that caused the unrounding of the round vowel /ʊ/ in most varieties of English (exceptions being most accents of Northern England3 and Midlands) and changed it to /ʌ/ (vowel in strut). So the round vowel of 'southern' changed to /ʌ/.
'South' was pronounced /suːθ/ until the Great Vowel Shift, which changed the /uː/ vowel to /aʊ/ (vowel in mouth).
- Both ⟨þ⟩ (þorn—thorn) and ⟨ð⟩ (eth) were used interchangeably to represent the th sounds in Old English. And ⟨ū⟩ represented the long U sound /uː/ (as in moon).
- The final e was pronounced up until late Middle English.
- That's why most northerners rhyme but with put.