To me, "going to the bathroom" means "going to use the toilet" although the word "bathroom" can mean both places, which is why I always try to clarify and say "I was in the shower" instead of "I was in the bathroom" as I think the latter would be interpreted as "I was in the toilet". Am I wrong?
As with any euphemism, the more common it is, the more likely it is that it will be interpreted non-literally unless appropriate context is given.
"To go to the bathroom" as an euphemism for using a toilet (or, more precisely, urinating/defecating) is common enough to make it to Cambridge Dictionary, so in my opinion you do risk being misunderstood.
"I'm going to the bathroom" is likely to be understood as meaning you're going to use the toilet, especially in American English. Even in British English, people are probably likely to interpret it that way, unless it's obvious from the context that you probably mean something different ("Sarah's had a shower - now it's my turn. Right, I'm going to the bathroom"). If you were going to have a bath or a shower, you'd be more likely to say "I'm going to have a bath" or "I'm going to have a shower". (British English prefers "have" here, where American English prefers "take".)
On the other hand, "I was in the bathroom" (in British English) could literally mean that you were in the bathroom (whether to wash or to clean your teeth or to use the toilet or for some other reason). How likely a misinterpretation is would again depend on context.
When somebody says that they are going to the bathroom, the most you can ever assume is the literal statement itself. Somebody could be going to the bathroom to use the toilet, or they could be going to the bathroom to use the sink. Or they could be going to the bathroom for any number of other reasons.
It's probably most likely that they are going to use the toilet, but there is no way to be sure unless you actually ask them. (Which could raise other social issues.) And different people will say exactly the same thing when they mean different things—just like anything else in English that's ambiguous.
Assuming anything in contexts like this always raises the possibility of being mistaken.
You will never know for sure what the person's literal intention is—aside from the fact that they intend to visit that room itself.
How the statement is understood depends to a great extent on the circumstances. If you are a dinner guest at a dinner party in my home, and you say you are going to the bathroom, or we are in a restaurant or a department store or the theatre, it can only mean that you are going to the loo (assuming you do not have a cocaine habit). If you are a house guest, and we pass on the landing, you in a dressing gown, then the same remark suggests you are planning to get ready for bed (or to get up, if it is morning).
But your question raises an issue. The statement is only an issue for two reasons: first, even (or perhaps especially) in these permissive times, many people do not like to refer directly to their bodily functions. We not like to say what they are going to do: ("I am going for a pee/wee", or "I'm off to defecate", or I'm off to have a bath/shower"). So we refer to the destination rather than the purpose. But also we do not quite like to speak directly of the destination itself as lavatory, toilet or loo. Instead Americans speak of a rest room. Bathroom is also a popular euphemism, or ladies, in my youth used to speak of going to "powder my nose", and even now I have heard people say "I'm just going to wash my hands".
So we are speaking not so much of the use of language to convey meaning and its use to avoid meaning (or at least certain meanings). Bathroom ore recently beginning to be be separated again in hotels and more and more homes.
But this is the world of hinting, as opposed to direct assertion.
It is taboo in many cultures to discuss using a toilet, whereas it is not to discuss bathing. The result is we can talk about taking (AmE) or having (BrE) a bath, so using the room named for that bath instead refers to using the toilet it also contains. And, to keep up this facade, we sometimes use the name “bathroom” for any room with a toilet despite many not actually containing a bath. (That may also be called a “half bath”, “washroom” or “powder room”, which also still avoid referencing the toilet.)