Yes letters are countable. But thy can also be thought of as abstract ideals with names:
This is V. V is a consonant. This is W which looks like two Vs put together.
Still in most contexts "It's a V" would be better than "It's V". Honestly I think the problem here is with the textbook, or rather iI suspect that, since I haven't seen the text.
As for the choice between "a" and "an" when an article is to be used, this follows the usual rule. When the next sound is a vowel, use "an" otherwise "a". Thus: "It's a V" but "It's an A". Note that this depends on the sound, not the spelling, and thus can vary in different dialects.