The presence of there may sound wrong and strange, but it is correct and natural.
Even more important, it is essential in the sentence, because without it, what would refer to something else altogether!
Let's take an example situation:
I do this boring job because I need the money.
Obviously, there is a reason to do this boring job.
If I want to ask about a reason to do this boring job, I can ask:
What is a reason to do this boring job? I need the money.
What refers clearly to the reason here, and I'll assume you'll have little problem with the formation of that question.
Now, let's say that we know the reason, but we want to clarify what the reason logically entails: doing the job.
What there is a reason to do is this boring job.
Here, what refers to this boring job, not to the reason!
I can even leave out this boring job, and I could tell you that there are things I do for a reason (like that job), and there are things to do without an obvious reason, which I may enjoy much more:
What there is a reason to do is useful, but terribly boring.
To repeat the exercise with your original sentence: there is evidence to believe something. Let's say that you found a knife in my hand while I'm standing next to a dead rabbit in pieces in my kitchen.
It is likely I cut up the rabbit, but there is no evidence that the rabbit was alive when entering my kitchen — the likely assumption is that I bought it good and dead from a butcher!
If we talk about the evidence — my location and my holding a knife — we simply refer to that as:
The fact I was holding the knife is evidence to believe (I cut up the rabbit).
What is evidence to believe (I cut up the rabbit) is that I was holding a knife.
However, if you are talking about me cutting up the rabbit, what should not refer to the evidence!
There is evidence to believe I cut up the rabbit.
What there is evidence to believe is that I cut up the rabbit.