We often say or write, "It is hard to believe."
Similarly, would it be correct if we say "He is hard to believe" to mean that whatever he is saying is hard to believe?
I am getting confused here?
“He is hard to believe” might be said of someone who frequently says things not easy to believe. But it is more common for someone to occasionally say something unbelievable†; and on such occasion, you might tell someone else “What he said is hard to believe”, or as suggested in Jay's comment, might tell the speaker “That's hard to believe”.
†Besides its literal meaning of “not to be believed”, unbelievable also has senses like “so surprising it is almost unable to [be believed]” and “Implausible or improbable”.
He is hard to believe
My interpretation of this is usually that someone is acting one way and saying something quite different. For example, if someone says, "I'll handle this," and then does nothing about it, that would be part of what I'd see as something that would make me think of someone as "hard to believe." In general, the idea would be that I don't have much faith in this person's abilities to get things done.
Hard to belive or Hard to swallow is an idiom that's commonly used when you suspect the authenticity of a story or account. For example:
Her story is hard to swallow or believe. Means that the story doesn't seem to be true, or at least that's what I think.