Is it correct to say:
I have both tea or coffee.
And does it differ from:
I have both tea and coffee.
"Or" certainly implies that you have either one or the other, but not both. You can't have both of them, yet have just one of them at the same time, so the first sentence doesn't make much sense. The second one is correct.
The conjunction both is used before two or more following things to stress that they all are included, whereas or always introduces an alternative.
Like you can't both have your cake and eat it, you can't have both something *or anything else -- it's always both one and the other or either one or the other