Short answer (tl;dr):
When Wilde was writing it was already literary and old-fashioned to use a relative clause without a relative word like who or that if the clause did not have its own subject. In modern standard English, the Original Poster is correct that the clause would need to be introduced by who or that. Oscar Wilde's sentence would be ungrammatical in modern standard English if written today.
However, this type of sentence was very common indeed in Shakespeare's day. And like many aspects of early modern English, this type of sentence lives on in regional varieties of English. So you may hear interesting sentences like these in various different situations
Full answer:
In modern standard English, when a relative clause contains an expressed subject, we can omit the relative pronoun (except in the case of whose). In such cases, the relative pronoun most often represents the object of the relative clause. However, when the relative pronoun is co-indexed with the subject of the clause (or actually is the subject, depending on how you interpret the grammar) it is not possible to omit it.
- That is the woman (
who) he loves.
- *That is the woman (
who) loves him. (ungrammatical).
In the example above above we see that in (1), where who is interpreted as the object of the relative clause, we can omit it. However in (2), where who would represent the subject of loves, it cannot be omitted. [Note however, In some non-standard dialects of English (2) would be perfectly grammatical].
However, this is only true about present day standard English. In early modern English it was very common to omit the pronoun is sentences such as (2), probably more common than to omit them in sentences such as (1). In Shakespeare's day the omission of relative pronouns was becoming rarer, but still occurred. He himself used bare relative clauses without pronouns reasonably frequently.
The Original Poster's Question - Modern English:
The Original Poster's sentence is from Oscar Wilde's novel The Picture of Dorian Grey. Wilde had a very literary style, and often liked to use fairly archaic language. Even in Wilde's day the sentence was a little bit archaic and had a literary effect:
I wonder who it was defined man as a rational animal.
This is a complicated sentence. For a start the matrix clause consists of the following main parts:
- Subject - I - pronoun
- Predicator - wondered - verb
- Complement - who it was defined man as a rational animal - subordinate open interrogative clause.
Because the interrogative clause is subordinate it does not have the same word order as it would in a main clause. Here is the main clause version of that same clause:
- Who was it defined man as a rational animal?
Further complicating matters is that this is an interrogative version of a special type of sentence called an it-cleft. Let's make it less complicated by turning it into a normal declarative clause instead of a question. We'll use 'X' to stand in for the question word who:
- It was X defined man as a rational animal.
We can substitute 'X' with 'the teacher' to make a complete version of the sentence:
- It was the teacher defined man as a rational animal. [ungrammatical in modern English]
We can make it simpler still by removing the extra information as a rational animal:
- *It was the teacher defined man. [ungrammatical in modern English]
This is a basic it-cleft construction that we have made easier to examine by making it a main clause and by making it declarative. A regular cleft sentence has four main parts: a dummy subject, the meaningless word it; a form of specifying BE, in this case the word was; a noun phrase (NP), in this case the teacher; and a relative clause, in this case defined man as a rational animal, which gives us more information about the noun phrase.
Unlike a normal relative clause construction, the relative clause and the noun phrase do not make a single large noun phrase. They are two separate parts of the clause. Nonetheless, the relative clause has the same rules that a normal relative clause has when it is attached to its antecedent. In the following examples we can see that the relative clause has a gap in the subject position.
- I don't like the teacher who [
he defined man as rational].
- I don't like the teacher that [
he defined man as rational].
- *I don't like the teacher [
he defined man as rational].
In modern English, the rules for relative clauses such as these allow us to omit the relative pronoun if the relative clause has its own subject, but not if it doesn't. For this reason (9) is ungrammatical. The following sentence in contrast would be fine:
- I don't like the man [the teacher defined
him as irrational].
Because the relative clause has its own subject, we do not need a relative pronoun like who or that here.
Note that because the relative clause in the Original Poster's example has been mangled and worked into a number of highfaluting constructions—it's been subordinated,it-clefted, buried in an interrogative—it looks quite posh and sophisticated. And for this reason may seem more grammatical than it really is.
The Original Poster's question: Wilde's English and others'
The Original Poster's sentence was already quite pretentious, literary and slightly archaic in Wilde's day, but relative clauses like these were still familiar enough to be considered grammatical in standard English.
Relative clauses like the one he used in the Original poster's (subordinate open interrogative it-cleft) sentence, are still perfectly grammatical in many regional varieties of English, but not in the standard Englishes, such as those which are taught in schools, or learned in language schools. They are not archaic any more, they are just not grammatical in these types of English.
Of course, that does not make them any less interesting or appealing when we see them in works of art from two centuries ago, or five centuries ago or when we see them used in those regional Englishes where they are perfectly grammatical. Enjoy them where you see them, but don't use them in your standard English!