I agree with you that both "having" and "giving" an interview can mean that you are the person being interviewed; the difference falls more on the side of whether you are the supplicant or the granter.
That is, if you are applying for a job, you are having an interview (and I think most would agree that your potential employer is giving it); but if you are a celebrity and you agree to answer questions for a magazine article, you are giving the interview. (I think in this case that hardly anyone would say that the magazine is having the interview, though.)
In both cases, the person who is in a position to decide whether the interview happens at all or not is the one who "gives" it.
If you want to be clear about who has what role within the interview itself, I'd suggest that the person answering the questions is "being interviewed" (as we can all agree already, it appears), while the person asking the questions is "conducting the interview".