There are always some people who are exceptions, but yes, native English speakers in general do clearly and easily distinguish these sounds.

I'm not a linguist, but from what I've read and seen it tends to be fairly common that native speakers of a language will easily distinguish phonetic differences that affect meaning, while ignoring those that don't. I'm guessing your native language doesn't distinguish these sounds: perhaps one of them isn't used, the same letter (or equivalent) can represent either sound, or which sound is used in a given word depends on the speaker's dialect.

I suspect most English speakers would have a similar problem learning a language in which the sounds of _s_ as in _sauce_ and _s_ as in _measure_ are distinguished (affecting meaning). As a native English speaker I can hear the difference if I think about it, but as far as understanding spoken English goes they're both the sound of _s._