Is the following sentence incorrect?
We asked him what was his name.
Should I put the verb after the subject because I'm using reported speech (ie We asked him what his name was) ?
Yes, you should put the verb after the subject. The sentence as you posted it is incorrect according to the usual rules of grammar. Note that the rules aren't as strict as sometimes assumed. If the subject is very long, the verb often stays before it. Compare:
We asked him what was the most important book in his collection.
We asked him what the most important book in his collection was.
Here the second version seems stilted and unnatural, and you should prefer the first version.
Even when the subject is very short as in your example, you sometimes hear the sentence such as you posted in conversational speech. But in the written language, it looks bad.
According to Michael Swan's Practical English usage the verb forms followed by wh- words need not be inverted.
(1)>We asked him what was his name.
(2) >We asked him what his name was.
Both the sentences are correct but the pattern in the order of a sentence is common. so the second sentence is the preferred form.