When a person or item is "hanging off (something)" in the sense used, it is attached to that (something) at or near its upper end, while its lower end is loose and can easily flap or flail about. In the example sentence, Scabbers is attached to the finger via his teeth, while his body and legs are unsupported. "Hanging off" generally has the connotation that the hanging item is not firmly connected and may be subject to an undesired, potentially damaging fall.
"Hanging on (something)" is a much more generic description, with the implication that the item being hung is firmly or appropriately attached to the (something). Generally, if a fall is possible, it is either very unlikely under normal circumstances or will not be damaging. You can hang a picture on the wall, but you can't hang a picture off the wall because the bottom of the picture won't be flapping free (it'll be against the wall, not moving). You can hang your coat on the coathook, but you can't hang your coat off the coathook (because the coathook is designed to have coats hanging on it). Also, in contrast to "hanging from" (see below), "hanging on" does not imply that the hanging object is mostly below the supporting thing; more likely it will be beside it or around it.
"Hanging from (something)" is similar to "hanging off" in that it implies that the lower end of the hanging object is loose, and more specifically that a significant portion of the hanging object is below the lower edge of the supporting something; but it doesn't have the same sense of an impending fall. I would say, for example, that "a banner hanging from the top edge of a building" implies that it was placed there on purpose and properly secured, while "a banner hanging off the top edge of a building" implies that it was placed on the roof, and accidentally got blown partway over the edge.
"Hanging by (something)" is used to describe the element connecting you to your anchor point: Scabbers is hanging OFF a finger, BY his teeth; a chandelier hangs FROM the ceiling, BY a chain. This is often used in the idiomatic phrase "hanging by a thread" to describe something that is in a very precarious situation where a disastrous fall is likely at any moment.
These are good general guidelines, I think; but @StoneyB does give some examples that contradict them, which is pretty typical of English.