I would like to check if my idea is correct. It sounds to me as if the first is when one has a question to be asked, while the second means the person is not sure.
Here are two cases:
In a classroom a student raises their hand(The teacher didn't ask anything):
- I have a doubt. What's ... ?
- I'm in doubt. What's ....?
The teacher asks that same student what is the answer for a question (The student doesn't know it and has nothing to ask):
- I have a doubt.
- I'm in doubt.
"I have a doubt" prepares the field for a question to come. That's why we use defined article "a", we know what is our doubt. I'm in doubt means one hasn't got the answer. So answering "I have a doubt" in the 1st case is better while "I'm in doubt" is better in the second.