Does this past perfect tense imply regret?

Question

Harry was remembering his trip to Diagon Alley - how could he have been so stupid?

It was on the corner of the street that he noticed the first sign of something peculiar - a cat reading a map. For a second, Mr. Dursley didn't realize what he had seen - then he jerked his head around to look again. There was a tabby cat standing on the corner of Privet Drive, but there wasn't a map in sight. What could he have been thinking of? It must have been a trick of the light. Mr. Dursley blinked and stared at the cat.
(Harry Potter)

The first past perfect tense, surely, shows Harry being self-critical. In the context, it seems the second one also has the same meaning, but I’m not so sure because of what. Would you let me know better?

Answer 1

No. The tense by itself does not imply regret.

The first sentence you quote does show Harry being self critical. He looked back at his past state of mind and wonders why he was stupid.

The second also shows Mr. Dursley being self-critical. He looks at his present state of mind and thinks he is hallucinating.

It is not the tense that causes these self-critical meanings, but the overall action of the individuals.

e.g.

"I was so stupid," said Harry, thinking back to his trip to Diagon Alley.

This gives the same sense of self-criticism, but a completely difference tense.

e.g.

"How could I have been misheard?" wondered Mr. Smith, thinking back to his discussion with Clarence.

This uses the same tense, but doesn't show self-criticism.

Answer 2

The second example What could he have been thinking of is closer to a literal question. There's an element of incredulity (in both examples), but the second example isn't carrying along any negative connotations such as so stupid.

Dursley simply saw something, and after realizing that what he saw was highly improbable (even impossible), looked again and realized his first impression was incorrect. There is puzzlement at what he 'thought' he had seen due to how far-fetched it was, but no sense of regret. He's got "... a trick of the light" to plausibly blame it on in any event.

Answer 3

They are parallel.

The declarative sentence underlying your first example is

He was being stupid.

Harry asks himself what deficiency led him to behave so stupidly.

That underlying your second example

He was thinking of [something improbable].

Mr. Dursley asks himself what irrationality led him to think he saw something which could not exist.

The tense is not relevant. You may just as well ask yourself, or another person:

How can you be so stupid?
What can you be thinking of?

Answer 4

To the extent that the present perfect is used with past events that touch upon the present in some way, the tense is certainly appropriate for expressions of present regret over past actions or failures to act. But to say that the tense implies regret would be to overstate the case.

"How" and "What" in such questioning exclamations often mean much the same thing, since "how" has a very broad range of meaning.

How could I have missed that turn off the highway!

What was I doing to have missed that turn off the highway!

The answer to both questions could be "I was daydreaming".