I came across this:
The phone rang when Tom was on the point of leaving.
As I usually use "at the point of leaving", are both possible?
I would use on, personally.
Google Books NGram Viewer suggests that both are possible, but that on is the more common of the two:
If I look at some of the actual examples of the at phrasing gathered by Google, it seems that it's sometimes used in a more literal sense (the banks of the bayou at the point of leaving the river), even though I see examples of its figurative use too.
So, both can be used. It's up to you to decide which you prefer.