There isn't a single correct answer. You could rephrase the questions:
Do you have three brothers?
Do you know it?
In either this style of wording or the style in the question, the speaker is looking for confirmation, a yes or no answer. You don't need to give a yes or no and then repeat the whole question in the form of a statement. In the form presented in the question, the speaker has already suggested the response.
Q: You have 3 brothers, correct?
A: Correct.
Q: You know it, right?
A: Right.
There is no need to elaborate beyond that, although you can is you want. If the asker is trying to start a conversation with small talk and you want to pursue that, a more expansive answer than "yes" or "no" would lead to conversation more effectively than a single-word answer. But repeating the question in the form of a statement would be more of a formal way to remove any ambiguity, not the kind of response that would encourage conversation. For example:
Q: You have three brothers, correct?
A: That's right. Charley is two years older than me and Nate and Sam are both younger. (Or whatever information might seem relevant and conversational.)
You wouldn't ordinarily need more than a one-word response unless the response was in the negative. In that case, you might want to clarify the way in which the question was incorrect. For example, if "you have three brothers" is incorrect, is it wrong because you have two brothers? Four brothers? Three sisters instead of brothers? You're an only child? You might get a clue as to what the questioner is trying to learn from the context of the conversation. But clarification would be more common with a "no" response.