TLDR
'South' and 'southern' have different vowels because they were affected by a phonological rule in Middle English called Trisyllabic Laxing (hereafter TSL). It was a process whereby a long vowel in a stressed syllable was shortened when it was followed by two or more syllables.
Explanation
Trisyllabic Laxing was a phonological rule in Middle English (and perhaps Old English) that shortened (laxed) a long vowel1 in a stressed syllable when it was followed by two or more syllables. It only applied to derived words (like divinity from divine and -ity) rather than underived words (like nightingale). At one point, it affected all the relevant words. Later on, it ceased to be a part of English phonology, though its remnants are still highly visible in Modern English. Some examples are as follows:
- insane /ɪnˈseɪn/ → insanity /ɪnˈsæn.ə.ti/
- profane /prəˈfeɪn/ → profanity /prəˈfæn.ə.ti/
- sincere /sɪnˈsɪə/ → sincerity /sɪnˈsɛr.ə.ti/
- serene /səˈriːn/ → serenity /səˈrɛn.ə.ti/
- impede /ɪmˈpiːd/ → /ɪmˌpɛd.ə.mənt/
- divine /dɪˈvaɪn/ → divinity /dɪˈvɪn.ə.ti/
- derive /dɪˈraɪv/ → derivative /dɪˈrɪv.ə.tiv/
- pronounce /prəˈnaʊns/ → pronunciation /prəˌnʌn.siˈeɪ.ʃ(ə)n/
- provoke /prəˈvəʊk/ → provocative /prəˈvɒk.ə.tɪv/
- holy /ˈhəʊli/ → holiday /ˈhɒlɪdeɪ/
As you can see, there are at least two syllables after the stressed long vowels, so they got shortened because of TSL.
There's a fairly regular pattern of the vowel change in the words I mentioned above. It can be summarised as:
- /eɪ/ → /æ/
- /ɪə/ → /ɛ/
- /iː/ → /ɛ/
- /aɪ/ → /ɪ/
- /aʊ/ → /ʌ/
- /əʊ/ → /ɒ/ (AmE: /ɑ/)
In Modern English, there are many exceptions such as words ending in -ness (e.g. mindfulness, loneliness etc) or later borrowings such as obese/ obesity etc.
The above changes happen in trisyllabic (three-syllable) or polysyllabic words, but 'southern' is not even a three-syllable word, why then is it pronounced with a short vowel?
The short vowel in 'southern'
The history of southern is more interesting. Wikipedia also mentions disyllabic laxing (when one syllable follows the stressed long vowel as in 'southern'), though the vowel in 'southern' is not the because of that. It is because of TSL.
The anomaly here is caused by historical sound changes. 'Southern' used to be a three-syllable word when Trisyllabic Laxing applied.
'South' was sūþ (or sūð)2 in Old English and it was pronounced /suːθ/. While 'southern' was sūþerne3 (or sūðerne) pronounced /ˈsuː.θer.ne/ ([ˈsuː.ðer.ne])
In Middle English, 'southern' was pronounced /ˈsuː.ðər.nə/. Then Trisyllabic Laxing applied to it and reduced the long vowel /uː/ to /ʊ/ (the vowel in put).
Then in Early Modern English, there was another sound change that caused the unrounding of the round vowel /ʊ/ in most varieties of English (exceptions being most accents of Northern England4 and Midlands) and changed it to /ʌ/ (vowel in strut). So the round vowel of 'southern' changed to /ʌ/.
'South' was pronounced /suːθ/ until the Great Vowel Shift, which changed the /uː/ vowel to /aʊ/ (vowel in mouth).
- The long vowels in the base words were then diphthongised by the Great Vowel Shift in Early Modern English.
- Both ⟨þ⟩ (þorn—thorn) and ⟨ð⟩ (eth) were used interchangeably to represent the th sounds in Old English (read this answer for details). And ⟨ū⟩ represented the long U sound /uː/ (as in moon).
- The final e was pronounced up until late Middle English.
- That's why most northerners rhyme but with put.
References: