If you have been told that recency is incompatible with past simple, you have been told wrong.
I finished it five minutes ago
is perfectly grammatical and normal.
I've finished it five minutes ago
to be less natural, though still possible.
The significance of a perfect construction is present relevance: recency is one manifestation of that, but not the only one.
In most cases, the choice of whether to use a perfect construction or not is not constrained by the objective circumstances (eg by whether an action is recent or not), but is a choice that the speaker can freely make in how they wish to present the temporal relationships. If the speaker uses a present perfect, they are choosing to represent the events described as having a present relevance; if they use a simple past, they are choosing to represent the events as completed.
But some expressions of time influence this choice. In British English, just used to strongly prefer the perfect to the simple past (when I was young I just saw him was an Americanism, and we would say I've just seen him; though I think both possibilities are available now). And to my ear ago prefers a simple past, though it doesn't rule out a perfect.