You've got it right. "Sour" is an accurate and appropriate description for a lemon. The opposite of that would be "sweet," although I'm not sure anyone would generally describe a banana as "sweet." That word is used for more extreme items, like chocolate cake. A banana could comfortably be described as "not sour."
That said, you're also right that there's a bit of ambiguity. "Sour" is a connotatively bad word in English, so it just depends on context. If you want to be clear, I would probably say.
The lemon is very sour. It's not bad, it's just very sour.
I know that's not a particularly elegant solution, but it seems like the best one I can think of. Although in most cases, I don't think that ambiguity would be much of an issue. People will know when you're talking about the taste, and "sour" is a pretty uncommon word to use to describe something as "bad."