Why questions about language are usually pointless: the answer is "because that is how the language is".
But I'll attempt an explanation nonetheless.
The perfect is used when we want to present the activity as having relevance to the present. Sometimes that relevance is about the result of the activity; other times it is because the activity is represented as continuing to the present, or in a period that is thought of as continuing to the present.
When you use a specific time in the past, that time cannot include the present. It doesn't make any difference how long or short the time is: last year or last Tuesday, or at 15:35 last Tuesday.
When you use a phrase like before or in the past, that is a period which can be regarded as extending up to the present (even "in the past" does that), so the perfect is possible.
Some periods can be either, with different meanings:
I haven't seen him today.
presents the time as continuing to now, so implies that there is still a possibility of seeing him.
I didn't see him today.
presents the time during which I might have seen him as finished, so implies that there is now no possibility of seeing him today.
Of your final three examples: they are not idiomatic for the same reason.
Note, however, that they become possible with a comma, making the time an afterthought:
I have done that, a long time ago.
This suggests that when I said "I have done that", I was thinking of it just "sometime in the past"; but I then qualified it with "a long time ago".