As Colleen has commentedcommented, the phrase is used more to give one's state or status, rather than to refer to one's location. In this sense, it's not really referring to the actual town the person will be out of.
Cancel all my appointments, I'll be out of town next week.
One could analyze this as being similar to other statuses, using other prepositions, such as away from keyboard or out to lunch, or even to more metaphysical statuses such as out of touch or out of sync. But not all statuses omit the: out of my depth, out of my league.
If you use a verb of travel, such as to go, it has more of the locative sense (talking about location):
Cancel all my appointments, I'll be going out of town next week.
But still, we don't use the definite article. It is probably a vestige of Old English, like at home or going home.
Note: English is not consistent about this. One person could say
I'll be out of country all week.
indicating a status, and another could say
I'll be out of the country all week.
which has a locative sense, and uses the definite article.
And few I know will say
I'll be out of county all week.
Probably because most people simply don't need to indicate that they will be out of the county (US states are divided into counties).