It mainly seems to be a reference to the literal physical act of waving your hand when telling someone not to worry about the details of something. I'm not sure if the gesture is international in use, so it looks very similar to the gesture made to shoo a bug away from your food.
Hand waving is used quite often in fantasy novels. The term "hand waving" is used if the author explains something "because magic." In other words they are saying - don't think too hard about it, just go with the premise I have set up.
An example would be the way many things work in the Harry Potter universe. Many things don't follow basic laws of phyics because Rowling has "hand waved" the physics by saying it's that way because people can use magic.
A more realistic example of hand waving would be a person omitting very technical or unnecessary details when giving someone information. I.E. A doctor tells a patient they have cancer. The patient asks how the doctor knows this. The doctor replies they did a biopsy. The doctor is "hand waving" many details about the exact chemical process used in the blood test. But the patient doesn't need to know those details, and probably wouldn't understand anyways.
The opposite of our doctor example also occurs quite often. The first example that comes to mind being on the playground! An excellent example of a presenter "hand waving" details would be: Child A claims X. Child B asks how A can know that. Child A replies "That's what my Dad says!" Child A has "hand waved" many details because he does not really understand what he is trying to explain.