Let (something) pass
To ignore, refuse to acknowledge, or choose not to retaliate against something someone else says or does.
"Those comments are very hurtful, but I'm going to let them pass because I know how frustrated you are."
"You have to let people's criticisms pass if you want to succeed in this industry."
Both "it" and "that" are grammatically fine here, and mean basically the same thing.
The only real difference is in how these phrases are habitually used*. "Let it pass" is a bit gentler, or perhaps more distant and less personal. "Let that pass" is a bit more direct, and perhaps slightly more confrontational. "It" could be anything, from any time... "that" is right here, right now.
Please note that this is a VERY subtle distinction, and native speakers won't typically think (consciously) about the difference, either when speaking or listening-- it would just be a matter of how the words feel at the time.
Footnote *: I will freely admit that "how a phrase is habitually used" can vary depending on region, or even depending on which group of friends you personally know. That said, I'm reasonably certain my explanation is correct for most people.