Surprise is in a class of verbs that also includes amaze, concern, delight, disappoint, encourage, elate, embarrass, impress, interest, shock, worry which can yield both "true passive" and, depending on who you ask, "quasi passive", "semi-passive", mixed passive" forms.
See, for example,
Oxford Dictionary of English Grammar, page 357.
Classification and SLA Studies of Passive Voice (pdf)
1 Mary surprised me.
2 I was surprised by Mary.
This can be analyzed as a true passive, where the actor in the active form becomes the agent in the passive form, and the object of the verb in the passive is used in subject position in the active.
Thus, your sentence
3 I am surprised by his response.
can be considered the passive form of
4 His response surprises me.
Semi-passives can exhibit adjectival properties (surprised modified by an adverb):
2a I was rather surprised by Mary.
The use of different prepositions with other verbs can exhibit the true passive sense:
6 His promise satisfied Sally.
7 Sally was satisfied with his promise.
8 English interested me.
9 I was interested in English.
10 I am surprised at how many have never heard of it.
can be recast as
11 How many have never heard of it surprised me.
Your other two sentences can be analyzed as passive forms of active forms:
12 To see you surprised me.
13 That some people study the past surprised me.
with the active of 13 being a short form of by the fact that....
Other analyses of I was surprised that some people study the past may be possible.
The take-home message is that the classification of English verbs is sometimes slippier than we might like.