"I am done V+ing something" means that whatever you are describing with the V (verb) was a long process. Buying, however, is a one-time act. One could spend a lot of time choosing a product, but the buying itself is not something that could be a lengthy process.
You could say,
I am done spending so much time on eBay looking for my perfect [something].
(Other examples for "I am done with": "I'm done putting up with this nonsense" (It means you have been putting up with something for a long time), "I'm done questioning and doubting myself," "I'm done wasting time on social networks", etc.)
Regarding your example, if you want to talk about the act of buying, it's enough to use Present Perfect:
I have finally bought the most expensive thing I needed to buy.
The phrase "I have no worries anymore" is perfectly grammatical. It's just that I don't see how not having a very expensive thing could be a cause for worrying. But that is a matter of personal philosophy :) A good phrase to use alongside "having finally bought something" is:
I'm all set now.
It means having everything you need and being prepared for anything that comes your way. Webster's Learner's Dictionary defines this meaning of "set" as "ready or prepared for something", with an example, "If we win the lottery, we'll be set for life. [=we will have everything we need for the rest of our lives]".
So, for example, if you buy a house, you are all set in the sense that you don't have to look for apartments or pay someone rent any longer.