The psychology of phonology is such that native speakers with no special training, and when they're not paying attention, cannot be realistically taken to 'hear' one allophone rather than another. Instead people usually just 'hear' the whole word and reconstruct what they hear.
For example, I am a native English speaker and I noticed one day that I consistently pronounce the 'b' in 'describe' as a 'v' for no apparent reason. I also pronounce the 'n' in 'only' as an 'm'. No one's ever mentioned it, and they only notice the difference when I point it out.
Take a look/listen to the following video:
http://www.youtube.com/watch?v=G-lN8vWm3m0
(Actually, I hope your native language has a distinct 'f' sound, or that you have learned how to pronounce the English sound in the typical way, with the upper teeth against lower lip; otherwise, the video might not make sense.)
I would focus more on comprehension and trying to emulate the accent you want to adopt, rather than particular allophones.