... And if the Dursleys were unhappy to have him back for the holidays, it was nothing to how Harry felt.
Normally, I see the phrase "it was nothing to" + sb, but here it's followed by "how Harry felt". So, I don't quite understand what it means. Is it a normal usage? What's the whole sentence trying to convey?