Plural agreement can raise questions of clarity. But context is important. When you're talking generally about groups of things, especially something like cars, you probably don't need the listener to know the exactly details of who owns how many things.
All of them have (a red car | red cars).
The meaning is immediately clear. Each person owns a red car. So you can really use either. I prefer the plural option in this case, because the subject is plural. If I wanted to use the singular a red car, I'd use a singular pronoun:
Everybody here has a red car. What a coincidence!
As a side note: it would be extremely unlikely that the speaker would mean to say that the group has a single car. (If that were the case, you would—or should—probably use the verb share: "All of them share a red car.")
When you need to specify that the groups do own multiple cars, you need to say that outright:
In the city's richest suburb, residents have multiple luxury cars per family.
In the example above, each resident owns more than one car. But in the example below, you can only assume that each resident has one car, but maybe more.
In the city's richest suburb, residents usually drive/have luxury cars.
In the example of the babies, things get a little tricky. Again, context is important.
Both of them have a baby boy/baby boys.
To be most clear, if each person has a single baby boy, you should say that.
Both of them have a baby boy.
These two examples might help shed some light:
We both have a boy at home. (We each have one.)
We both have boys at home. (We have multiple.)
We both have only boys at home. (Emphasis on boys.)
I found a lovely thread in English Language & Usage that thoroughly addresses the question of distributive plurals and arrives at the same conclusions as above.