It really depends on context. If your uncle said "I don't think your sister's working hard enough on her homework" and you said "I agree", it's hard for that to be interpreted as impolite. However if your uncle was criticizing you, then it might come across as flippant: "I don't think you are working hard enough", "I agree."
Even then, whether or not it's impolite depends on your tone of voice and body language more than the words you say. Truthfully, "I agree" is so neutral that it's hard to find a better alternative for politely letting someone know you agree with them.
You might say "sure", "definitely", or simply "agreed" if you don't have anything to add. Alternatively you might say "I've been thinking the same" or "you're absolutely right" if you're worried that a one word reply might sound rude.
Ultimately, the best trick to avoid sounding condescending is to simply not look down on people. If the other person worries that you're not being respectful enough based on how you talk (especially as a non-native speaker!) then it's up to them to ask you to clarify what you've said.