In statements like "it's letting up" the word "it" functions as a dummy subject called an expletive (defined as a syllable, word, or phrase inserted to fill a vacancy (as in a sentence or a metrical line) without adding to the sense).
In your example of "it's letting up", "it" refers to adverse weather conditions, usually rain. You could say "the rain is letting up". Like other pronouns, "it" doesn't have to be something previously mentioned if it is really obvious what you are referring to, such as a person looking up at the rain and assessing it.
However, "it" does not always need to refer to any specific noun or pronoun. An expletive serves a grammatical purpose to fulfil the requirement for a subject in English sentences, particularly when talking about things like the weather, time (eg "it's late"), distances (eg "it's a long way"), and other situations.