Why is it not past perfect for the word 'left'?

Question

Check this paragraph:

I had been working in the garden for an hour when I heard the phone ring. It was Jamie. He was driving down to London on business. I had gone out in the garden when he left, so he had only been gone an hour. I knew he had some problem.

My question : why "left"? To me all verbs in the last sentence "had gone out", "left" and " had been gone" happen before I heard the phone. So all should be in past perfect because I heard the phone is the reference time.

I know sometimes with since, you don't use past perfect:

I saw Peter yesterday, I hadn't seen him for a while since he left for Germany."

The word 'since' in that case indicates the beginning of the action. But when is not since and in that paragraph, first the person went out in the garden then Jamie left. Or may be I should understand the sentence like this:

He had only been gone for an hour since he left and at the time of the leaving I was already in the garden.

Answer 1

In English it is common, and in some cases obligatory, for subordinate clauses to shift their tenseframe this way, in order to express a distinct relationship to the Reference Time (RT) established in the head clause.

In this case, for instance, your RT is as you say the time at which the phone rang. A past perfect signifies a state current at RT which arises out of a prior eventuality; for instance, I had gone out in the garden signifies that when the phone rang, the author was in the garden. There the prior eventuality is not focal but merely mentioned.

To speak focally of that prior eventuality, the author employs a simple past. In this context, in a subordinate clause (when he left), this is not understood as a deictic tense (one ‘pointed to’ from speech time) but as a relative tense: an eventuality prior to RT.

If the author had written when he had left the entire tenseframe would be shifted to the moment of Jamie’s departure, which is perfectly grammatical but just adds another level of ‘processing’ when the discourse reverts to the RT tenseframe with he had only been gone an hour.

Answer 2

A very important use of the Past Perfect Tense is that it is used to clarify which event happened earlier when two actions were completed in the past.

I had gone out in the garden when he left.

It means, "When he left, I was already in the garden."

"I had been working in the garden for an hour = so he had only been gone an hour". This is possible only if the speaker started working in the garden just before Jamie left. So the use of the verbs 'left' and 'had gone out' is appropriate.

Perhaps it is not the right thing to do when you have guests in your house. Maybe when the speaker went to the garden Jamie took the cue and left the house.