Or should it be, "I was wondering if you were sleeping"? Also is it right to say, "I was worried you would be sleeping and opening the main door would disturb you"?
I think what you're after is
I was wondering if you might be sleeping
The latter would should also be might in your second sentence.
A more verbose alternative:
I was wondering whether or not you were sleeping
A negated version should also suffice in context:
I was wondering if you weren't sleeping...
And an inversion of your latter sentence feels more natural to me:
I was worried the main door might wake you if you were sleeping
I was wondering if you would be sleeping.
This sentence doesn't make much sense and it's better if you don't use this as it might confuse its readers. That sentence is not complete. However in:
i was wondering if you were sleeping.
This is a complete sentence and makes sense. You should always try to use simpler form of sentences for better understanding of the reader.
Both sentences are grammatical, and have slightly different aspectual meanings.
I was wondering if you would be sleeping...
I was wondering if you were sleeping...
In the first sentence, the speaker refers to a thought he had in the past; he was contemplating what he might discover if he should open the door; he might find that you had been asleep.
In the second sentence, the speaker refers to a thought he had in the past; he was wondering if you were asleep, so that his opening the door would disturb you.
would be refers to the possibility of discovering you asleep; it is forward-looking, here a looking-forward set in the past. were refers to the possibility of your being asleep at the time.