Neither, actually. “Where” would generally be the right word for the reason you mentioned. However, the status of the place is an essential quality, and the most natural word to use to open a defining relative clause is “that.” That being said (see what I did there) “where” is defensible because “that” has already been used in the sentence, and “which” (or, in this case, “where”) is often used even when the relative clause is defining. This exception helps avoid awkward constructions such as “that that doesn’t kill you makes you stronger” (example borrowed from grammarbook.com).