The idiom is: to be used to something.
Present: People who aren't used [or are not used to] to wearing green [etc.]
Simple Past: People who were weren't [or were not used] to wearing green [etc.]
PP: People who haven't been used to [or have not been used to] wearing green [etc.]
Often, English language learners confuse to be used to something with the verb to use something and the defective imperfect form: I used to [do something].
The form didn't use is the simple past tense of the verb use and is not related to the idiom given at the beginning of this answer. The verb use works like this in the SP:
He didn't use the paper I left on top of the printer.
The form of I used to [do something].
I used to play tennis=defective imperfect tense.
Negative*: I didn't use to play tennis, the negative is not so common.
For repeat actions in the past.