To quote a TRomano's comment:
mutt is often used as a synonym for a dog of unknown pedigree or the offspring of such a dog, the "opposite" of a purebred dog, so to speak
Because of this meaning, "mutt" has a negative connotation. As such, it may be used to describe a dog for which the speaker has some kind of negative feelings. If a purebred dog bit your hand, for instance, you may yell "You mutt!".
"pooch" is simply an alternative word for "dog". It's not used very often nowadays but almost everyone would understand what you mean. The idiom "screwed the pooch" (meaning the subject made a mistake) is the most common usage.