I am a non-native speaker of English so I am trying to get used to every phrase I come across while hearing and reading. As a result, I have watched many American movies and so I have heard the phrase "speak English" plenty of times. Nevertheless, I still don't get the meaning of it. For example, here are several quotes from the movie "The Avengers":
"Bruce Banner: Well, if he could do that, he could achieve heavy ion fusion at any reactor on the planet.
Tony Stark: Finally, someone who speaks English."
"Steve Rogers/Captain America: Hey, speak English.
Tony Stark: You see the red lever? When I tell you to, pull it. That will give me enough time to get out."
Aren't they speaking English? Or is there another hidden meaning behind it that I don't catch?
Thanks in advance.