This expression X has nothing to do with Y can mean two things (at least):
- If Y is the word "you," it can mean X does not concern you. When used in this way, the expression can be used in a somewhat unmannerly way, as if to say, "X is none of your business!"
Ted: I think maybe you're spending too much money this Christmas.
Ed: What I put in my shopping cart has nothing to do with you.
- It can also mean X is not the cause of Y. In the following dialogue, assume Ned is a salesman who had no sales last month:
Ned: Our team won Team of the Month. We should go celebrate!
Ed: Ned, you had nothing to do with this award.
(That could be regarded as a harsh remark, but the phrase can also be used in a reassuring way, to say that something is not your fault:)
Jed: What a calamity! I feel so bad that this happened.
Ed: Don't fret about it, Jed. You had nothing to do with this.
Back to what you were trying to say:
4am has nothing to do with your English.
I guess I can sort of see how this might mean the time of day doesn't affect your English, but I'd probably not word it that way. Instead, I'd say something more along the lines of:
Your English is great, even at 4am.
As for the other way around:
Your English has nothing to do with 4am.
That one seems a bit harder to catch the meaning. I might word that like this:
Your English isn't affected by the time of day.
Your English is good round the clock.