Sam addressed the problem that was reported by Tom.
is probably the meaning you intend. Tom reported a problem, and Sam is talking about it.
Sam addressed the (reported by Tom) problem.
sounds convoluted, but could have some meaning.
If Tom reported many problems that were not really problems, and Sam was tired of hearing his complaints, then the problem is Tom. So Sam addressed the problem of too many annoying things reported by Tom. So:
Sam addressed the (annoying things reported by Tom) problem.
But this is a stretch (trying really hard to make any sense) at best.
the problem
is your direct object so you need to put it after your verb, andrepirted by Tom
gives more information about the problem so it should come afterthe probleml
.Reported by Tom
is called relative/adjective clause and cannot become before the word or phrase it explains.