He is known to me.
Known here is a past participle functioning as an adjective. It is not a passive construction
What your sentence means is
To me, he is known.
Compare this with an adjective:
He is funny to me.
equals
To me, he is funny.
Sometimes it is very difficult, if not impossible, to determine whether a given sentence is a passive construction or not.
See "I am surprised": passive voice or adjective?"I am surprised": passive voice or adjective?
The classic case is
The window was broken.
Is broken an adjective or a passive verb form? It could be either, And without context it's impossible to say. Adding an agent clarifies that it is a passive construction:
The window was broken by me.
Also consider:
The store is closed every night at eight.
Closed could be an adjective or a part of a passive verb construction.
Adjective:
The store is not open but closed every night at eight when I walk past it.
Passive:
The store is closed every night at eight by me. This involves taking out the trash and locking all the store's doors before I go home.
Some verbs have both an active and stative sense. Known is one of these.
I know this song.
Could a passive be formed:
? This song is known by me.
To a native speaker, that sounds pretty weird.
You found a sentence that uses know in a passive construction:
"The Earth was known by Galileo to be a sphere".
(Active equivalent: Galileo knew the Earth to be a sphere".)
I worry him.
is a perfectly good active sentence, with a direct object; but still it makes a questionable passive:
?? He is worried by me.
sounds very strange.
Last, know has an old-fashioned meaning, equivalent to lie down together (and have sex).
Adam knew Eve and she conceived and bore him a son.
can be changed into a passive:
Eve was known by Adam, and she conceived and bore him a son.
This is what is called to know someone 'in the biblical sense, because this use of know was made famous by the King James translation of the Holy Bible.
