Deriving from specifically this use, we use pump as a verb. It can mean up and down, it can also mean back and forth. I suppose it might be used left and right as well, though I don't recall a time it did in my experience. This is where we get expressions like pumping iron (moving weights up and down or back and forth so as to build up muscle) and all of the things that derive from it.
In British English, it's definitely a handle in that situation. A lever would be something used in a more basic mechanical way, and we use lever as a verb, but less often for something that's part of a bigger machine. I know that Americans use lever for more parts of machines, but I don't know if they use it for pump handles.