“Sorry?" barked Hagrid, turning to stare at the Dursleys, who shrank back into the shadows. "It's them as should be sorry! I knew yeh weren't gettin' yer letters but I never thought yeh wouldn't even know abou' Hogwarts, fer cryin' out loud! Did yeh never wonder where yer parents learned it all?"
This is from Harry Potter. I think "fer cryin' out loud" is "for crying out loud", but I don't know how to relate it to the context. How should we understand it in the context?