English speakers distinguish these sounds almost perfectly. Certainly with well over 99% accuracy. As pointed out in another post here, any phonemes that create a difference in meaning in a language (in a substantial number of environemntsenvironments) will be clearly and reliably distinguished by native speakers.
If you are a Japanese speaker planning to speak English with speakers whose first language also distinguishes /l/ from /r/, then it is ESSENTAILESSENTIAL that you learn to make these sounds so that they are distinguishable. Even if you don't remember in every case, you need to be able to make these sounds completely distinct. It's even better if you can train yourself to hear the difference. This is a much more difficult task, but it's doable. Many very good Japanese speakers of English find it difficult to hear the difference. However, all very good speakers of English can produce the sounds correctly.