No, upside down does not modify a note. It modifies read.
Looking at the passage in context (typed up from my UK edition, which has some small differences):
‘Gallopin' Gorgons, that reminds me,’ said Hagrid, clapping a hand to his forehead with enough force to knock over a cart horse, and from yet another pocket inside his overcoat he pulled an owl - a real, live, rather ruffled-looking owl - a long quill and a roll of parchment. With his tongue between his teeth he scribbled a note which Harry could read upside-down:
Dear Mr Dumbledore,
Given Harry his letter. Taking him to buy his things tomorrow.
Weather's horrible. Hope you're well.
From context, we can see Hagrid doesn't intend for Harry to read the note at all; the intended recipient is Dumbledore. From this, the most natural reading is that Harry is looking at it upside-down.
The other reading is grammatically possible but pragmatically questionable.