Maybe. Hand me that ____ implies particularly the act of giving it to you by hand, and as a native US English speaker I would only use it if the person already had it in their hand or if it were very nearby (i.e. it was "to hand").
For example, if the pan were sitting on the counter right in front of my spouse, but she were not actually holding it, I would probably still say "Hand me that pan." But if it were in another room, hand would sound inappropriate. I don't think there is a strict rule for how far away something is, but I might say "Hand me two eggs" even if they were in the refrigerator, but not "Hand me the colander" if it were in a cabinet in another room.