First, I'll put on a veneer of scientific-ness and consult a corpus!
I searched the Corpus of Contemporary American English (COCA) for alien. That had 8783 results, which is a lot more than I intended to look at, so I clicked the
SAMPLE: 100 link to get a random sample of 100 results. (The order of the first 100 results is not random by default, so that's a helpful thing to do.)
Looking at those results, here's what I noticed:
- Alien by itself usually means space alien.
- Alien is used sometimes in legal contexts, usually in terms like legal resident alien or illegal alien, though it does occasionally appear by itself.
- When people are described as "aliens", it's often put in quotes as though it's an unusual term.
Although these results are based on a corpus of published material, these observations fit my intuition about how the term is used in American English. Personally, as a speaker of American English, if I heard your sentence:
Aliens are surprised when I tell them I don't have a television.
I'd assume you meant space aliens, and this assumption is supported by the corpus results I looked through.
And now for my opinion:
My personal impression is that the term alien is very "other-izing". It's a term that makes whatever you're describing feel very different and unfamiliar. My feeling is that it's not appropriate to apply to people outside of specialized contexts precisely because people aren't "other" enough. We're all humans, after all!
If my impression is correct, it might follow that xenophobes (people who fear the "other") are more likely to use this term to describe people. And for both of these reasons, I think it's possible that people could feel marginalized by the term or take offense to it. So to me, it feels like it's best to avoid using alien this way.
My guess is that this is true outside the U.S. as well, but I don't really know.