First, a small but important correction: me in Say/tell/explain X to me plays the same semantic role as bare me in Tell me X—both me's are Recipients—but it has a different syntactic role. Bare me is an Indirect Object, but me in to me is not an object of the verb—it's an object of the preposition to. Some grammarians call this me an oblique.
So a first-order answer to your question is “because tell is ditransitive—it is permitted (but not required) to take two objects, while say and explain are monotransitive—they are permitted to take only one object (each).”
That of course is just a name for the phenomenon, not a true because; but you're rarely going to get actual becauses in matters of this sort. Language is a democracy; we've been voting on these uses of those three words continuously for over a thousand years (well, only about 600 in the case of explain), and the only one of the three that got a majority of votes for ditransitivity is tell.
In other words, it's that way because we like it that way.
(My personal theory is that this Reign of Linguistic Terror came about shortly after the Norman Conquest, when the suppressed witenagemot, the Englisc proto-parliament, met in secret and launched a centuries-long project to make the Englisc language unintelligible to non-native speakers, so no one would ever be tempted to conquer the Anglo-Saxons again. But the evidence is wholly circumstantial.)