The interesting thing here is that the "wh-clause" "what I don't know" can be hypothesized to be derived from the canonical sentence "I don't know what", which happens to be the other one you're asking about. Yet the meaning is different.
The difference is in the position of the wh-word. In wh-clauses, the wh-word--such as who, what, or when---moves to the front. If it doesn't move to the front, then you do not have a wh-clause.
So, "what I don't know" is a wh-clause, whereas "I don't know what" isn't.
The wh-clause is a nominal clause which denotes whatever the wh-word refers to. "I'm waiting for what I don't know" is somewhat nonsensical, but it has a valid grammatical form. We can make a very similar sentence which makes sense:
I want to learn what I don't know.
The "what I don't know" wh-clause serves as a noun, denoting that which I don't know (and that is what I want to learn).
On the other hand:
I'm waiting for I don't know what
is an instance of transformation; perhaps extraposition. (The extraposition examples resemble this situation). That is to say, we can regard it as a rearrangement of the canonical sentence, where the "I'm waiting for" part moves to the front:
I don't know what I'm waiting for.
Note that in this sentence, the "wh-clause" is "what I'm waiting for", not "what I don't know"
So basically the difference between your two sentences is the role of what. In one it is "what I don't know" and in the other it is "what I'm waiting for" (rearranged by a transformation which moves the "I'm waiting for" part away from "what" out to the front of the sentence).
Lastly note that although "I'm waiting for what I don't know" is nonsensical (how can you wait for something that you don't know?), the same words in the same order can express this:
I'm waiting. For what, I don't know.
This has the same meaning as "I don't know what I'm waiting for", or "I'm waiting for I don't know what", or "I'm waiting; I don't know for what".