Yes, you nailed it correctly. The comma is what makes the difference. As oerkelens says, It was a pity is an independent sentence and if that is the case, the previous sentence should end before it begins.
This tricky question uses comma and then asks for the word to put. In simple words, the first part of the sentence gives us the gist and the latter is simply adding the emotion or sympathy to it. What we call supplementary relative clause is used here and it is often set off by putting comma and then which. One good way to identify this is removing the latter part of the sentence which was pity. Even after removal, the sentence's meaning is intact and this does not happen in case of integrated relative clauses. That said, removing which part in such examples will make the sentence looking ungrammatical/weird.
OxfordDictionaries gives an example of a restrictive relative clause: The coat that/which Dan had on yesterday was new.
Remove which part and it won't look a proper sentence.
So, to answer you in short, the word is which because it introduces supplementary relative clause i.e. additional meaning. The comma before the word confirms it.