This is how I read this rather confusing clause.
The speaker is metaphorically comparing his memory to some sort of cup with liquid in it. Think of a cup of coffee, tea, or wine, where the grounds, leaves, or sediment fall to the bottom. Those are "the dregs". The speaker is saying the dregs of his memory is where deeply buried, half-forgotten memories end up. He'll never let the memory of what this man did fall into those dregs; he'll keep it fresh in his mind, floating at the top of his memory cup, for as long as he can until he has the chance to get revenge.
So you basically had it: he's saying he'll delay his revenge, but he'll never let the memories fade; he'll keep on remembering why he desires this revenge, until the day he gets it.