I understand the correct pronunciation of "of" is /əv/. However, in fast speech, can I pronounce it as /əb/ (which sounds very close to /əv/)? I feel like it's easier to say it that way, (upper and lower lips just touch), rather than the lower lip has to move backward a bit so that the back of the lower lip touches the upper teeth.
I have never heard of a native speaker using /b/ in "of". To English speakers, /v/ and /b/ don't generally sound very close.
Simplification of the pronunciation of "of" can occur in fast speech, but the usual way this happens is dropping the /v/, leaving just /ə/ (as in the example given by Andrew in a comment: "a cup of coffee" /əˈkʌpəˈkɑfi/. I believe this is less common before vowels.
No, not really, because they sound very different to most English speakers.
The v sound in of is a voiced labiodental fricative. That means your vocal cords move while you're making the sound (it's voiced), it's made with the lower lip and the upper teeth (it's labiodental), and it's a kind of buzzing sound (it's a fricative).
The b sound in ub is a voiced bilabial stop. That means your vocal cords move while you're making the sound (it's voiced), it's made with both lips (it's bilabial), and you stop the airflow through your mouth (it's a stop).
To me, the big difference is the fact that v is a fricative and b is a stop. You can keep making a fricative until you run out of air: uvvvvvvvvvvv... But you can't keep making a stop - it "stops" when you make it: ub!
If you pronounce of as /əb/, you won't sound like a native speaker. If you say it very quickly, people might not notice, but if your goal is to learn a language, you should learn to pronounce it as the natives do.