"The town's inhabitants/ residents." Is there a difference when used in this kind of sense?
As Jasper says, the term resident confers a notion of right (not necessarily a legal right, but that most easily fits), that inhabitant does not necessarily imply.
Both can be used in that phrase you supplied - but one may be preferable over the other -depending upon the actual subject, or topic, of discussion.
I think it would be more idiomatic if you write "residents of the town" and "inhabitants of the town".
I looked up both on google and saw the usage of "residents of the town" is much more common than "inhabitants of the town" in this sense.
I think two word inhabitant sounds a little bit formal and maybe scientific sometimes.
And another difference between them in terms of usage is that, while the word inhabitant can be used for animals and humans, the word resident only can be used for humans.
Hope this helps a bit