A lot of people say "faith" is some kind of blind belief in something without any proof. Actually, the primary definition in many dictionaries is "confidence or trust in a person or thing". You only usually place trust or confidence in something or someone if they have a solid reputation, or they have proved trustworthy in the past, so really faith by definition does have a basis in something.
In this staircase illustration, the implication is that an entire staircase exists but you can only see as far as the first step. If your vision had limited distance and you could see the first part of a staircase, that would probably give you reason to believe that the rest of the staircase was there, you just couldn't see it.
So with this illustration Martin Luther King was encouraging people to take the first figurative step towards something, even if they didn't know what the rest of the steps might be or what they would look like, and to have faith or confidence that the rest of the steps would come.