It is easy for him to make her nervous.
In this sentences 'to Infinitive' is done by him.
He was terrified for them to ask too many questions.
But I am not sure in this sentence who is subject of asking. I was aiming to say he got some fear from numerous questions of them.
If 'to infinitive' comes after a 'for noun', does it always means 'to Infinitive' is done by the noun??
*He was terrified for them to ask too many questions.
Your sentence is ungrammatical, so I put an asterisk (*) to denote it. Your choice of preposition is wrong. You should use "of".
Other than that, it would be a valid sentence without of them, but a slightly awkward one with.
To answer your main question, no, that is not the case.
The questions were too hard for Jim to solve.
Here, as you expected, Jim is solving questions. However, in a sentence such as
I was sad enough for them not to say anything in my brother's wedding.
The doer is the subject I, not them.
There would be a potential for ambiguity in these cases, but usually the meaning of the sentence is so clear that you don't need double-checking.
As for the wording of your sentence, I'd suggest using the pattern too + adj. + [ . . . ]1 + to + verb as it's commonly used for contexts similar to yours — a person being prevented from doing something for some reason. The intensifier too conveys your meaning perfectly.
He was too scared/terrified of them to ask too many questions.
1: In the blank, you may use "of"/"for"/"with" etc. and a pronoun, depending on how the elements of your sentence relate to one another. In your example, it's of them.
