There are four parts to this question
the primary intention of the question, which I think is somewhat hidden, is that there is a tendency to informally call any language error a 'grammatical error'. But this usage is not preferred (i.e. to say "Is this punctuation grammatically correct?" is an error of word choice. Grammar almost always refers to just syntax and word formation - using that word for punctuation or word choice sounds really weird.
But the question sets things up by example to expect an answer which enumerates distinct parts of language study (giving an opportunity to label them appropriately, implicitly answering the first part but for other parts of language study). I mostly separate them into the following:
- phonology - how to produce sounds, both those sounds that just don't exist in your own language, and also that vague idea of 'accent'
- morphosyntactics - how to arrange or modify words, grammar (all those other things aren't really called grammar)
- semantics - what you mean by the choice of words or syntax
- vocabulary - word choice - what you use a dictionary and thesaurus for
- culture (sociolinguistics) - politeness, expectations, assumptions
- spelling and punctuation (orthography)
The point here is that 'wrong' can be applied differently according to the domain a rule lives in. A sentence can be syntactically correct but semantically nonsense, similarly you can string a small set of words together 'you Piano crush above run' and you know unequivocally what to do despite the utter rulelessness of the syntax.
This all assumes that there are errors. You might well wonder how one can question the existence of errors given that here on ELL all the questions are about, fixing errors. We all want to get things right. But native fluent speaking is different from non-native learning. And even though there is usually a single standard to follow, there are errors in trying to speak like that standard and then there are 'errors' which are really just
- variants within the standard (pail vs bucket)
- acceptable informal versions (defecate vs bowel movement vs poop)
- alternate varieties of the same named language that are just following a (slightly) different set of rules (just consider the difference between British and American English: 'going to hospital' is wrong, wrong, wrong in AmE, but the way you're supposed to say it in BrE).
- socially undesirable variants - like using ain't or 'double negatives' or 'f' for 'th'(which is really identical to the previous section, just socially charged)
- style choices that are enforced formally but are really made up by one guy once and others just started to follow (injunctions against split infinitives, end of sentence prepositions)
So sometimes there are real errors: "He not is no thief" (no one ever says that), and other times there are 'errors' "He ain't no thief" is OK under very informal circumstances and people do say it, just not in formal circumstances or in writing.
The point here is that 'wrong' can be applied in different ways, sometimes as really wrong/incorrect/erroneous and sometimes it's just being 'judgy'.
And lastly, sometimes you can have a sentence which is 'correct' under all the supposed rules of all the sections mentioned above, but just ... it turns out that no one says it that way. Maybe a language teacher made it up for the third lesson in a beginner's class, or it's how Shakespeare would naturally have said it, or there's some idiom that people tend to use instead. It feels funny to say that it's wrong because no real rules have been broken. It's just too bad because no one talks that way. "long time no see" is 'broken' English but you'd prefer it over the nominally correct "It has been a long time since I have seen you".
(side note: sometimes you can give a sentence which on first sight looks totally ungrammatical, but then with the right situation/context and appropriate intonation/emphasis the sentence is interpretable as grammatical and meaningful. This may be the case, but for language learners, the context is probably not what was meant. If that particular context was meant, well that sentence is gonna sound really weird and should probably be reworded because, my last point, people just don't say it that way.)