According to the rules, we need to use "the" when we refer to peninsulas: the Balkan Peninsula, the Taymyr Peninsula, the Breton Peninsula, the Crimean Peninsula. However, is there any rule for the use of articles when the word "peninsula" is ommited? Why the Crimean Peninsula becomes Crimea, but the Balkan Peninsula retain "the" in the Balkans? How is it defined? For instance, does the Taymyr Peninsula become Taymyr or the Taymyr?
The names of these peninsulas seem to take the form of adjectives. Breton is the adjective related to the region of Brittany (Bretagne). Crimea and the Balkans are the names of regions - https://en.wikipedia.org/wiki/Balkans. I had to look up the Taymyr Peninsula, but it seems Taymyr/Taymyria is also the name of a region.
I think you have "the rule" the wrong way round.
The name "Crimea" or "The Balkans" comes first. There is no perfect rule for deciding if a place name has "The" as part of the name, but I'll note that plurals are more likely to have "The" (the Balkans, the States) but there are exceptions (The Levant)
Then you form the name of the peninsular using the place name as a determining adjective
Let's go to a peninsular.
The Balkan/Crimean/Cornish/... peninsular.
And "The" is included because the name determines which penisular.
So you don't form "the Balkans" from "The Balkan peninsular" but the other way round. "The Balkans" don't retain "the", they always had it.
As for Taymyr, it doesn't conventionally have "the".
As your link says:
Language is always changing. Fifty years ago, Argentina was known as “the Argentine,” and the Falkland Islands always had the definite article in front of it. But Argentina is now perfectly standard and you may even hear people say “I come from Falkland Islands.”