What your name is? is ungrammatical. You must say What is your name?
The reason you cannot say What your name is? is because that word order makes the listener hear what as a relative pronoun. A listener hears What your name is? not as a question, but as a fragment of a sentence like this one:
I know what your name is.
In this sentence, the word what functions as a relative pronoun: it makes what your name is into the object of know. Every interrogative pronoun (who, what, where, when, why, how) can also function as a relative pronoun. The word order tells the listener which role the interrogative/relative pronoun is playing.
Here are some examples to illustrate how the word order changes (or stays the same) when you switch between declarative statement, relative clause, and question:
- Your name is Jamius. / I know what your name is. / What is your name?
- Terry is ready now. / I see who is ready now. / Who is ready now?
In line 1, the word order reverses to form a question, because what stands for a subject-complement or object. In line 2, the word order is the same in all three sentences because who stands for the subject.
- Whose name is Jamius?
In line 3, the question's word order is the same as the declarative sentence, since Whose name corresponds to the subject of the declarative sentence.
There are some other important things to know about English questions, like what to do with auxiliary verbs, the need to add an auxiliary verb (usually do) when the verb is anything other than be, and the fact that you can use the word order of a declarative sentence to make a question express surprise or emphasis (for example, Your name is WHAT??). But the above explains why What your name is? doesn't work as a question in English.