A better rule would be to use "quantity" (or "amount") when the included items are considered as a single group and "number" when they are considered as discrete items. For example (an asterisk indicates a poorly formed sentence):
An uncountable noun is never considered to include discrete items, so we don't use "number"1:
The pharmacist bought a large quantity of medicine.
The pharmacist bought a large amount of medicine.
*The pharmacist bought a large number of medicine.
With a countable noun considered as a single group, we don't use "number":
He bought a large quantity of drugs.
He bought a large amount of drugs.
*He bought a large number of drugs. (Incorrect for this meaning, but see below.)
With a countable noun considered as a collection of discrete items, we don't use "quantity" or "amount":
*The quantity of drugs in his body was between two and five, according to the coroner.
*The amount of drugs in his body was between two and five, according to the coroner.
The number of drugs in his body was between two and five, according to the coroner.
Therefore, we could use "quantity", "amount", or "number" depending on whether the Porta Potties or apartment buildings were considered as a single group or as discrete items.
1 There are various definitions of "uncountable". My definition includes words such as "news", so we could say, for example, "a large amount of news" but not *"a large number of news".