See cold: "7. adjective - A cold person does not show much emotion, especially affection, and therefore seems unfriendly and unsympathetic. If someone's voice is cold, they speak in an unfriendly unsympathetic way." – user3169
desire : hot :: lack of desire : cold. To "turn cold" means all of the ardor goes away. – Tᴚoɯɐuo
So, the meaning is that men are may be strongly (sexually) attracted (to a woman) who is distant from them, but when they get close (physically or emotionally) to the woman they become less (sexually) attracted to her. For women the opposite is true.
In Tom and Jerry, there is a scene in which the Dog has been given sleeping medicine by Tom, and his eyes show "Out Cold" meaning fast asleep. That's not directly related to this sense.