Harry was remembering his trip to Diagon Alley - how could he have been so stupid?
It was on the corner of the street that he noticed the first sign of something peculiar - a cat reading a map. For a second, Mr. Dursley didn't realize what he had seen - then he jerked his head around to look again. There was a tabby cat standing on the corner of Privet Drive, but there wasn't a map in sight. What could he have been thinking of? It must have been a trick of the light. Mr. Dursley blinked and stared at the cat.
The first past perfect tense, surely, shows Harry being self-critical. In the context, it seems the second one also has the same meaning, but I’m not so sure because of what. Would you let me know better?