First note that your first example is incorrect. You should write
Mary cheated on her exam. If her teachers called her out on it, she'd just deny everything.
Note that the phrase "called on her" is very common in a classroom setting. It refers to the teacher selecting someone to give an answer verbally, for example, to a question posed to the whole class. This is distinct from "called her on."
Calling someone out on something is a phrase that implies that a person is doing something wrong or shoddy and it's pretty obvious to everyone that it is wrong, but it's not obvious that someone will point out the problem. Normally this is not what I would select in this case unless the cheating was really obvious to everyone.
Calling someone on something, I think, is taken from poker, where you call someone's bluff. In other words, there is dishonesty involved and to call someone on it means to test them in such a way that their potential dishonesty is revealed. I have seen people use it in a little more generally, as you have, but I think that's not really the proper application of the word.
Neither of your two examples are completely wrong, but normally I would say something like
Mary cheated on her exam. If her teachers caught her, she'd just deny everything.
or
Mary cheated on her exam. If her teachers confronted her about it, she'd just deny everything.