Both wordings are correct ways of describing different things.
Normally, "has been going off" would be used with the implication "has been going off and on"—meaning "has been on and off intermittently" during the specified time period. In fact, the full expression, "has been going off and on," is quite a bit more common than the shortened but implied form "has been going off [and on]" as a way of describing multiple interruptions in normal service.
In contrast, "has been off" indicates simply an uninterrupted period of being off: there has been no electricity (in this case) for the past two hours.