The line is
The calls from Harvard hovered at our potential, which was right around zero
This means that the amount of calls from Harvard hovered at the some level on an imaginary graph. What level? The same level as the level of their potential, which was right around zero.
You can also imagine this "hovering" as the "hovering" of an indicator hand:
The indicator needle hovers at the zero level. It goes slightly up, then goes slightly down, but generally stays in one place. Imagine that the instrument measures not the speed of a car but "calls from Harvard".
Examples of usage:
The speedometer needle hovers steadily at just over 100 miles, yet Engineman Earl Gilbert sits relaxed at the controls. (Popular Mechanics)
Mark lifts his foot off the gas pedal and the speedometer needle hovers at 55 mph as they round the bendin the road. (Google Books)