In addition to what Cookie Monster said,
It's often the case that when someone says "the city", it's understood that they're referring to a nearby urban area that they both know about.
For example, imagine you live near some city, maybe Austin Texas. If you were at a bar and you met a stranger, and asked them where they lived. They might say
Oh, I live in the city
It would be understood in context that they don't just mean they live in some city, but that they live in the nearby city of Austin.
They might also say something like
I live in the suburbs
That is, outside the city, but not in the country. But even this phrase implicitly refers to "the city", because suburbs are connected to a city. So you know that they are referring to the suburbs of Austin specifically, not just any suburbs.
In addition, some regions have a single "the city" that a lot of people talk about. Some people on the east coast of America will refer to the city of New York as "the city", and might even capitalize it as "The City", just because it's such an important city. But that usage is less common.