Some people only use "whom" in formal writing, but some people always use it where it is grammatically appropriate. This tends to depend on education level, norms within certain social groups, etc., but the choice is up to you.
This is correct, and feel free to use it:
Those guys whom you talked to are looking for you now.
Many people use "who" in place of "whom" at all times:
Those guys who you talked to are looking for you now.
Some people will use "that":
Those guys that you talked to are looking for you now.
You can also drop the pronoun entirely:
Those guys you talked to are looking for you now.
Different native speakers will have different habits, but all of the above forms are common enough. Using "whom" can be seen as pretentious by some people, but it is not unidiomatic.