You're right: you have identified an ambiguity in English.
When a noun is formed from a verb, like "care", it is sometimes ambiguous whether an "of" complement is semantically the subject or the object of the verb.
Other examples:
The experience of poverty ("poverty" is the object of the verb "experience")
The experience of people in the past ("people" is the subject of "experience")
The choice of colours is soothing ("colours" is the object of "choose")
The choice of the majority of visitors was not to take a tour ("the majority of visitors" is the subject of "choose").
In this case only real-world experience lets us conclude that "the children" is the object of "care", but "strangers" is the subject.