Your book is right, whereas your opinion is wrong.
As a matter of fact, the sentence #1 is not correct grammatically.
You use the simple past, not the present perfect, to talk about times such as yesterday, last week, in 2014, 2 years ago.
So the direct speech of the indirect speech "Tom said that he had had a strange experience the day before/the previous day" is the sentence #2.
Per backshift rule, you change the past simple into the past perfect simple in indirect speech.
You also change the present perfect simple into the past perfect simple, but the fault in the sentence #1 is that you have used "yesterday" with the present perfect simple. Please look at the following sentences:
Tom said, "I have had a strange experience today/this week". (direct)
Tom said that he had had a strange experience that day/that week. (indirect)