Take CAMBRIDGE DICTIONARY, for example:
DEFINITIONS:
SUBTLETY: a small but important detail; the quality or state of being subtle or hard to notice
NUANCE: a very slight difference in appearance, meaning, sound, etc.
3 NICETY: a detail or small difference that is only obvious after careful thought
EXAMPLES:
- SUBTLETY:
All the subtleties of the music are conveyed in this new recording.
Listening to the interview, I was impressed by the subtlety of the questions.
- NUANCE:
The painter has managed to capture every nuance of the woman's expression.
Linguists explore the nuances of language.
- NICETY: They spent a lot of time arguing about legal niceties.
We don't bother with all the social niceties here.
In each of these examples, these three terms seem to convey the sense of "barely recognizable details or distinctions that the average layperson is unable to distinguish that only intelligent or experienced people can make out"