The word "of" has many, many senses, and often a single occurrence of the word evokes more than one sense at the same time. In this case, I think two senses occur simultaneously: (1) indicating an object removed, and (2) indicating passivity in regard to an action. (These are senses 14 and 50 in the OED.)
Here are some other verbs used with "of" these two senses:
The Duchess was cured of insomnia.
The photographers robbed the Duchess of her dignity.
The parking lot was deserted of cars.
The jury cleared the defendant of all charges.
According to all recognized principles of law, … an individual bereft of reason, either by the act of Providence, by accident, or by his own act, with one exception, is not responsible, legally, for his acts committed while in that condition. (Source.)
The Indians had innumerable ceremonies and rites, … such as ritual bathing, or opacuna, as they call it, which was to bathe in water in order to be cleansed of sins. (Source, slightly edited.)
The best way to understand "of" (and English prepositions in general) is not as having a completely independent meaning of its own, but a smear of many meanings that exist only in connection with familiar verbs and phrases where it's combined. "Of" isn't governed by rules as much as it evokes different familiar phrases when used in different contexts.
The reason you don't say "the Duchess delivered a baby" is because "deliver" (in regard to babies) usually means to assist the mother in giving birth, as a midwife or obstetrician. The word "deliver" in this sense frames (very strangely!) the assistant as the agent of the birth: the assistant "delivers" the baby, while the mother is viewed as the patient, that is, the person who is acted upon. The phrase "delivered of a baby" accords with this conceptualization of the mother as the passive participant in the "delivery", that is, removal, of the baby. "Delivered of" sounds a little archaic today, though—hence appropriate for talking about royalty.
"Deliver (of) a baby" is actually a narrowing of a now rare sense of "deliver" meaning removal of a burden. "Deliver" comes from the same Latin root as "liberate", meaning to set free. The mother is set free of the baby.
Here are a couple examples from the King James bible:
Before she travailed, she brought forth; before her pain came, she was delivered of a man child. (Isaiah 66:7, describing an impossible sequence of events: a woman giving birth before going into labor.)
A woman when she is in travail hath sorrow, because her hour is come: but as soon as she is delivered of the child, she remembereth no more the anguish, for joy that a man is born into the world. (John 16:21)
Here's an example (from 1822) of "delivered of" not in regard to a baby:
…for death will soon clear away all our old assessment cases, and leave, I have no doubt, in a few years, the one-half of Glasgow wholly delivered of its compulsory pauperism. (Source.)
In other words, when the poor people die, the city will be relieved of the burden of paying taxes ("assessments") to care for them.