I was just wondering if the term "speech-impaired" was offensive. I hear people say it a lot, but I am not sure if it is offensive to those who are mute.
"Speech disorder" and "speech impairment" are common formal terms for describing a wide range of conditions. They are formal and rather medical terms. Not offensive.
Formal English tends to prefer to use the noun rather than the adjective. That is you would say "A person with a speech impairment", rather than "a speech-impaired person".
Of course nearly anything can be used offensively, so context matters.
Muteness is only one form of speech impairment, and not the most common.
The term "speech-impaired" is generally considered outdated and offensive by many people. It is important to use language that is respectful and accurate when talking about individuals with communication disorders. A more appropriate term to use would be "mute" or "nonverbal," as these terms accurately describe a person's ability to produce speech without implying that there is something wrong with them.
See also https://ncdj.org/style-guide/ as there is a paragraph on similar term, "hearing impaired":
Many dislike the terms because “hearing impaired” describes a person in terms of a deficiency or what they cannot do. The World Federation of the Deaf has taken the stance that “hearing impaired” is no longer an acceptable term. NCDJ Recommendation: Avoid using “hearing impaired” or “hearing impairment.” For those with total hearing loss or who identity as a member of the Deaf community, “deaf” is acceptable. Others prefer “hard of hearing.” It is best to ask your sources what they prefer.
While it is not a slur and indeed is a term used by, for example, educators or speech therapists, you would not use it casually in front of someone who had some sort of speech issue anymore than you would call attention to some visible physical defect. It is tricky: If I had migraine headaches and someone with a third person present asked, observing me sitting with my eyes closed, Oh, another migraine? I would probably not mind (although I might mind someone revealing a health condition to that third person).
But if I limped, I would certainly mind someone casually bringing this up even if it was obvious to everyone that I did, especially jokingly.
In general, health problems in the United States are only carefully discussed since they might have an effect on one's employment. It is a personal matter and only if it was absolutely necessary to mention it, as in maybe your group is making a presentation and you are trying to decide who will speak in front of, say, a client. Then you would very tactfully say something like, "The client is not familiar with the way Joe speaks and I think while Joe did great work on preparing the presentation, Jane should probably be the speaker -- if there are technical questions, Joe will be ready to explain things."
Note that I did not even use the term "speech impairment" -- that I think would be more embarrassing to Joe than just saying, "the way Joe speaks."
On the other hand, if Joe's color blindness might interfere with his reading slides, I think it would be ok to ask Joe directly if he felt his color blindness might affect his ability to present. I actually don't know why color blindness or short-sidedness is probably not at all embarrassing to most (but still be careful) but a speech impairment is -- I guess kids tease each other about speech impairments, imitating them while I never encountered anyone being teased about color blindness although severe vision impairment is also something that is probably sensitive to many.
Bottom line, no matter what terms you use, be very careful about bringing up any sort of chronic illness or disability. Very easy to get fired for doing so and if in doubt and it seems really necessary to discuss someone's health issues or disabilities, ask someone like your boss or in human resources.
(I realize that you did not ask specifically about workplace, but that is where the stakes are likely to be highest.)