Yes, let's is indeed simply a contraction of let us, and that means that whenever you can use let's, you can use let us.
But that is not the whole story!
The expression let's (or let us when used in the same way) is idiomatic; it means something different than you would think by just looking at the dictionary definition of let.
You correctly mentioned that let's play is a proposal or encouragement to play something.
Now, the normal meaning of let is different, as you noticed as well. It means "to allow or enable something". When you say let us into the house, you are using the imperative. That means you are giving someone an order.
Let us into the house means make it possible for us to move into the house.
It is important to understand that you do not use the contraction let's in this situation! *Let's into the house is not correct.
Now, in some cases, a sentence may be read in two different ways, and that can be confusing. Look at the two sentences:
Let's go in.
Let us go in.
They can both mean "I propose that we go in", but the second one can also mean "Allow us to go inside!"
The actual meaning of the second sentence depends on context:
It's getting cold outside, let us (let's) go in!
We need to search your house. Let us in!