Gngram finds that ages ago is much more common than millennia ago.
In the example you gave in your comment to your own question, you give this example:
I read this book millennia ago.
Here ages ago would sound much more natural.
The example in the body of your question might work with either expression depending on what that
something is. For example I would say:
There were a lot of poplars on this road ages ago (not millennia ago).
There were a lot of stone inscriptions in India millennia ago (not ages ago).
However, I do think one is free to emphasise as much as they want, in spoken language or in literature. I would definitely be ok with:
There were a lot of visitors in my father's house millennia ago, now it's so deserted!