When combined with the implication that what the fisherman spat out into his hand was worms, the phrase "you have to keep your worms warm" becomes the punchline of the joke but there's no special meaning to the phrase... it means, literally, that the man believes that warm worms make for better fishing.
What makes the joke funny is the absurdity of the guy keeping the worms in his mouth which, by most Western considerations, would be disgusting and unpalatable, even if the solution gave you a bucketful of fish... particularly as keeping worms in your mouth isn't the only way to keep them warm, it just happens to be the solution that the fisher decided to use.
Here's another version of the joke:
Take The Bait
It was a cold winter day, when an old man walked out onto a frozen lake, cut a hole in the ice, dropped in his fishing line and began waiting for a fish to bite.
He was there for almost an hour without even a nibble when a young boy walked out onto the ice, cut a hole in the ice not too far from the old man and dropped in his fishing line. It only took about a minute and WHAM! a Largemouth Bass hit his hook and the boy pulled in the fish.
The old man couldn't believe it but figured it was just luck. But, the boy dropped in his line and again within just a few minutes pulled in another one.
This went on and on until finally the old man couldn't take it any more since he hadn't caught a thing all this time. He went to the boy and said, "Son, I've been here for over an hour without even a nibble. You have been here only a few minutes and have caught about half a dozen fish! How do you do it?"
The boy responded, "Roo raf roo reep ra rums rrarm."
"What was that?" the old man asked.
Again the boy responded, "Roo raf roo reep ra rums rarrm."
"Look," said the old man, "I can't understand a word you are saying."
So, the boy spit into his hand and said, "You have to keep the worms warm!"
This version of the joke plays out the fact that the boy was speaking unintelligibly twice until he spat out the worms and repeats the phrase in a way that can be understood... and manages to follow the comedic "rule of threes"