Let's say each city has its own city-wide speed limit.
- The speed limit varies between cities.
- The speed limits vary between cities.
Definition 1 for "vary" of this dictionary reads:
to be different in different situations
which implies that only one single characteristic, without it belonging to a specific thing, is being talked about, so sentence 1 should be the correct one.
But, definition 1a of the same dictionary reads:
if two or more things vary, they are different from each other
which implies that two or more characteristics, each belonging to a separate thing, are being talked about, so sentence 2 seems to be the correct one. What do native speakers think?