I was told once that one of the two options is more rude than the other, What is exactly the difference?
If for some words to be rude you mean you do not intend to offend someone but these words backfire you, then "I don't care" is more rude than "I don't mind". For, that Mary does not mind John's act means that she loves him too much to mind that act, while that Mary does not care John's act means that she does not love and she does not hate him at all so that she does not mind that act at all. In sum, the two sentences imply different motives, out of love and out of indifference.
But if you simply wish to tell the truth, then there is no rudeness contained in the two sentences. For, in this context when you say you don't care then you mean exactly you don't care.