This sentence is incorrect. It's acceptable in speaking, but not good at all in writing. Without a specific context around this sentence, it's a poor choice for an otherwise excellent series like the Grammar in Use books.
In this sentence, someone had previously said something like, "I went to the factory two weeks ago". This sentence is now being reported, so according to the normal rules of reported speech, the reporting verb ("claim") and the times in the sentence get shifted to the past. So, "went" is shifted to "had gone". The expression "two weeks ago" always and only means "two weeks before the present time", so "two weeks ago" should be shifted to "two weeks before/earlier."
But you can also report speech without shifting the times back, including the reporting verb. This way of reporting speech asserts the speech being reported is still true now.
So if, for example, "he" said those words this morning, then his "two weeks ago" still refers to two weeks before the present time, so everything can remain in the same time:
"He claims that he went to the factory two weeks ago."
It doesn't make sense to have "ago", which is rooted in the present time, in a sentence about something said in the past. The meaning is that someone in the past claimed to have gone to the factory two weeks before the present time, rather than the time he was speaking.