He opens his mouth, but the words don't come out
This leaves it open that even though the words currently aren't coming out, then may in the future or when he tries a different approach.
He opens his mouth, but the words won't come out
This more strongly suggests that the problem is unfixable. No matter what he tries, the words won't come out. Not now, not in the future.
Note that this is not a stated fact, but simply a stronger implication compared to using "don't".
Secondly, keep in mind that this is a song. In lyrics, grammatical correctness can be overridden for lyrical flow. Especially for rap, easy pronunciation is essential to speaking quickly yet being easily understood.
Is it ok to say like this and is there any specific rule that says in which situations one can use won't when they would normally use don't/doesn't?
The distinction between "won't" and "don't" is effectively the speaker's opinion (in regards to how permanent this statement is). Logically, there is no rule that can define whether what the speaker believes is actually correct or not.
However, there is a guideline here. If you're writing a story in which the speaker is convinced the problem is unsolvable, "won't" is a better choice. If the speaker thinks that the problem is only temporary or an easy fix may exist, then "don't" is a better choice.
But neither options are definitive. It all hinges on how you want to portray the speaker.