Some students took the exam.
Most of them passed.
These two sentences (clauses) can be combined into one sentence by converting one of them into a relative clause. I'm basically going to present a mechanical method of doing this, a method that in general might be helpful, but maybe sometimes it might not work out so great. But anyway . . .
(CAVEAT: I'm assuming that what sentence #2 means is that most of them had passed the exam.)
First, let's look for stuff that's common between the two sentences. There is an obvious pairing -- the pronoun "them" of the 2nd sentence and the noun phrase "some students" of the 1st sentence. That pronoun "them" is equivalent to those students who had taken the exam.
So if we relativize the 1st sentence (into a relative clause) and shove it into the 2nd sentence, we can get:
- A. Most of them [who took the exam] passed.
We can now replace "them" with the noun "the students", and get:
- B. Most of the students [who took the exam] passed.
This version #B seems to be the answer that your quiz was looking for.
(Aside: Also acceptable is "Most of the students who had taken the exam passed".)
(Aside: But notice that version #B seems to be missing some info that was in the original pair of sentences: that is, that the original pair implied that some students had not taken the exam, and that implication is missing in version #B.)
We can also try to relativize the 2nd sentence (into a relative clause) and shove it into the 1st sentence:
- C. Some students, [most of whom passed (the exam)], took the exam.
It seems to me that this version #C would probably need a more specific type of context for it to make sense; while it would be relatively easy to make up a context that would let version #B make sense. Also, notice that version #C is using a supplementary relative clause, instead of an integrated one like that in version #B.
There's also another possibility: that is, the exam itself can be used to be relativized. It has been assumed that the 2nd sentence is basically saying that "Most of them passed the exam." And so, here's two more options:
The index "i" links up all the stuff that's related to "exam": the gap "__(i)" is linked to "which" and that is in turn linked to the ultimate antecedent "exam".
Notice how version #D is saying something quite different from what was said in the original version: that is, version #D could have an interpretation where there is an implication that there are some students who had passed the exam but had not taken that same exam (for a context, think university scholarship athletic program--which has very recently been in the news). This last interpretation could be made more obvious via "Most of the students passed the exam which some of them took". (Though, note that the quantifier could be removed from the relative clause, to give something like "Most of the students passed the exam which they had taken", which seems to basically have the same meaning as version #B.)
It seems that version #E basically has the same meaning as that in the original version, and the same meaning as in the quiz's answer (version #B).
Now, this analysis has assumed that the original #2 sentence meant "Most of them passed the exam". It is possible that there could be a context where sentence #2 could mean something different, such as "Most of them passed the course".
Hopefully this is enough info to help you with respect to relative clauses.