Does "which" indicate that I have just one car?
I should sell my car, which I use very little.
Does "that" indicate that I have more than one car?
I should sell my car that I use very little.
Does "which" indicate that I have just one car?
I should sell my car, which I use very little.
Does "that" indicate that I have more than one car?
I should sell my car that I use very little.
Neither of the two options precisely limits the number of cars you have; I would not suggest using either which or that in this case, because both words are referring back to the object in the earlier part of the sentence with ambiguous terms. Of the two options, which is closest to the implication that you only have one car, but it does not absolutely rule out the possibility of a second vehicle.
A more precise phrasing might be "I should sell my car, since I use it very little."