As said above, the question is imprecise and depends on context. Others have agreed that it is a sentence, so I will try to assume the opposite opinion.
It is not a sentence, it's an exclamation. In school I was taught to answer in full sentences. Saying "Five!" was not acceptable. "Yes." is in a different category than "Yes!" as in "Yes! I did it." As was pointed out above, it might be a minor sentence, but I'd say it's still not a fully qualified sentence. If it is not ambiguous and you don't have to shout(!) to get the point across, just use a full stop instead.
In mathematical predicate logic, false and true are predicates on their own. Now I could go on about zeroth-order logic [1], but I don't want to make too many mistakes. The point is, could you write a complete text only of exclamations? Doubt it! I mean, I doubt it, not that you should, but the interpretation as imperative would actually be the most common use of the exclamation mark. Otherwise it's more like "wow ... did you see that?" (not fully quallified on it's own). So it depends on context.
[1] https://en.wikipedia.org/wiki/Zeroth-order_logic
Edit: Of course, "Wow!" can also be understood imperative.
Anyhow, a minor sentence seems rather colloquial. In any formal setting where the distinction between a sentence or something else would matter, colloquial styles would be mistaken.
Edit to address @J.R.'s comment: In mathematical predicate logic, "true" as well as "true -> true" (implication) are predicates, but "true ->" is syntactically incorrect. I allude that the exclamation mark likewise expresses semantic meaning that requires some consequence. The meaning is implicit and hence not an explicit sentence i.e. formally incomplete. The exclamation mark distinctly symbolises the difference to a sentence. Otherwise, I don't see a difference between "Yes." and "Yes!". The distinction is indicative vs imperative.
Of course, this is an argument by analogy and therefore debatable. Also, that's highly formal, therefore use of a single word exclamation could be precluded for style reasons alone.