You are correct that you might simply use had, which is the past tense. Here, the writer used the past perfect tense, which is typically used to describe something that happened before something else. In this case, it's not really clear what that something else is, but it's probably the incident or circumstances that are the subject of the conversation.
What makes it even more confusing is when the verb is had, the auxiliary verb is also had in the past perfect tense.
Perhaps another example would help:
I had enough of his shenanigans. (past)
I had had enough of his shenanigans. (past perfect)
In the past perfect, it's implied that something else occurred before I had enough of his shenanigans. For example:
By Tuesday, I had had enough of his shenanigans.
So to answer your question, since it's not clear what that "something else" is when using had had, there is really is no difference between the two. Perhaps the author used it to add emphasis to the statement.