The accepted answer here correctly explains that the "o'" contraction is short for "of the", but it does not explain how it came to be in the first place.
Prior to the introduction of clocks, there were other indicators of time, like celestial bodies. Even when clocks were introduced, very few people had them, and common people still used other means to tell the time. (It should be noted that the concept of counting time in hours precedes clocks). It was customary to refer to e.g. the fourth hour of the day based on the time elapsed since sunrise.
However, with the introduction of clocks, it became necessary to distinguish time told from one means to the time told by a clock, as they would not always correspond. (The sun comes up earlier in summer and later in winter). Therefore, to make the distinction that time referred to was being told by a clock, one would say "of the clock".