When the term "overdose" is applied to drugs or alcohol, the meaning is that the user has taken enough to cause serious medical problems. When said about somebody else, e.g., "Jenny overdosed on heroin", often the implication is that the drug killed them.
But "overdosed" can also be used figuratively to mean "had more than I should have", e.g., "This afternoon I overdosed on chocolate." Or even "Last weekend we overdosed on episodes of the Simpsons."
Your example "I think I might have overdosed on vitamins" is likely intended figuratively. It is possible to cause medical problems by taking too much of certain vitamins, but I doubt that applies in this case.
The word "overdone" means "done to excess", and doesn't have the implication of serious medical harm or death. The phrasing "overdone on" doesn't sound right in most contexts; it would usually be "overdone it with [something]" or "overdone the [something]". So the original sentence you mentioned could be rephrased as "I might have overdone it with the vitamins", or "I might have overdone the vitamins".