Good question, but it's worth pointing out that glitters in this context is a verb, not a noun. The subject of the sentence is all.
When all is used as the subject of a sentence, it functions as a pronoun, and it can function as a singular or a plural pronoun. From NOAD:
all (pronoun) [with clause ] the only thing (used for emphasis) : all I want is to be left alone.
(used to refer to surroundings or a situation in general) everything : all was well | it was all very strange.
All that glitters is not gold
is roughly equivalent to
Everything that glitters is not gold
and the verb tense agrees.
All who eat are not gluttons
is roughly equivalent to:
All the people who eat are not gluttons
so there is no problem there, either.