For something simple like clothing, there really aren't any dedicated nouns for an unwanted feature. We might use any number of adjectives meaning "forbidden" in certain contexts, but none consistently. More often we'd leave the avoidance as understood in context or we would use a phrase that moves the avoidance into the verb part of the sentence rather than as a noun.
For example:
"... with epaulets, avoid those. I would never..."
Or perhaps as a statement of preference in the same place:
"... with epaulets, I hate those. I would never..."
That said, if you're really wanting a noun, then in addition to Tᴚoɯɐuo's suggestion of "show-stopper" and your own "deal breaker" you might try no-no.
no-no: something that people are not supposed to do because it is not proper, safe, fashionable, etc.
Here the word describes the action of either buying or wearing a shirt with epaulets.
However, none of these words are really right in the context of an unwanted feature on a relatively low cost, unimportant item like a shirt. If it were a car, or a house, or a prospective mate, or a business contract, or a behavior, then yes. You could get away with any of these occasionally in this context, but if you tried to use these words frequently for this purpose it would sound strange.