It here refers to brain. You look for hint in the same sentence, try to identify singular noun or compound noun that can be replaced by it in the sense that it needs to rest.
All the singular nouns or compound nouns in the sentence are: (the action of) going to bed and brain. Obviously, the meaning tells you that the brain can't rest if it is stimulated too much by playing video games before going to bed. The action of going to bed cannot rest, but here only the brain can.
EDIT. The best answer chosen by the asker is actually wrong in this sentence meaning, as I explained in the comment section of his/her answer: if it in the example refers to the situation, then the sentence has to be "making it hard for the brain to rest", otherwise you would be saying "making the situation hard to rest", which is meaningless. I am aware of the linguistic phenomenon that sometimes we use it or there to be a dummy subject to fill the sentence structure, however in this case it doesn't function in the same manner.
I highly recommend that the asker unchecks the other person's answer as the best/correct one in order not for other English learners to be biased towards and acquire inaccurate information.