The hyphenated phrase “128-bit” is, all together, an adjective. The hyphen indicates that it is being used in this manner. As an adjective, it modifies the noun “key.” These kinds of hyphenated adjectival phrases are very common, often with units of measure (“128-bit,” “12-inch,” “100-gram,” “5-hour,” and so on), and they always use the singular form of the unit in their construction.
Meanwhile, in “128 bits,” we have an adjective—the number “128”—and a noun, “bits.” Because the number is not 1, the noun is plural, hence “bits” and not “bit.”
Since I did mention that the above construction is common with units of measure, I should note that when you use the International System of Units (SI) abbreviations (m, g, s, etc.), those aren’t pluralized. This is an SI rule, though, not an English-language one—in fact, you see that traditional English units often do pluralize even unit abbreviation, as in “5 lbs.” for “5 pounds” vs. “1 lb.” for “1 pound.” But we don’t do this with the hyphenated construction, for instance “a 5-lb. dog.”
Which to use depends on context and style. For instance, if you were differentiating between two different things of a given measure, like a “5-lb. dog” vs. a “5-lb. cat,” the adjectival-phrase version, as I’ve used here, is much easier to read and write. Otherwise you’d have to say something like “the dog that is 5 lbs.” vs. “the cat that is 5 lbs.,” which is very unwieldy. In other cases, it’s easier to not do that, for example, “we have two keys, one with 128 bits and another with 256.” But it largely comes down to style and what exactly you’re trying to say, which will sound or read better.