There are three parts to this question
the primary intention of the question, which I think is somewhat hidden, is that there is a tendency to informally call any language error a 'grammatical error'. But this usage is not preferred (i.e. to say "Is this punctuation grammatically correct?" is an error of word choice. Grammar almost always refers to just syntax and word formation - using that word for punctuation or word choice sounds really weird.
But the question sets things up by example to expect an answer which enumerates distinct parts of language study (giving an opportunity to label them appropriately, implicitly answering the first part but for other parts of language study). I mostly separate them into the following:
- phonology - how to produce sounds, both those sounds that just don't exist in your own language, and also that vague idea of 'accent'
- morphosyntactics - how to arrange or modify words, grammar (all those other things aren't really called grammar)
- semantics - what you mean by the choice of words or syntax
- vocabulary - word choice - what you use a dictionary and thesaurus for
- culture (sociolinguistics) - politeness, expectations, assumptions
- spelling and punctuation (orthography)
This all assumes that there are errors. You might well wonder how one can question the existence of errors given that here on ELL all the questions are about, fixing errors. We all want to get things right. But native fluent speaking is different from non-native learning. And even though there is usually a single standard to follow, there are errors in trying to speak like that standard and then there are 'errors' which are really just variants on the standard or informal versions or socially undesirable variants but are just follow a (slightly) different set of rules (just consider the difference between British and American English: 'going to hospital' is wrong, wrong, wrong in AmE, but the way you're supposed to say it in BrE). So sometimes there are real errors "He not is no thief" (no one ever says that), and other times there are 'errors' "He ain't no thief" is OK under very informal circumstances and people do say it, just not in formal circumstances or in writing.
And lastly, sometimes you can have a sentence which is 'correct' under all the supposed rules of all the sections mentioned above, but just ... no one says it that way. Maybe it's how Shakespeare would naturally have said it or there's some idiom that people tend to use instead. It feels funny to say that it's wrong because no real rules have been broken. It's just too bad because no one talks that way. "long time no see" is 'broken' English but you'd prefer it over the 'correct' "It has been a long time sine I have seen you".