You've got it exactly right; remember has two different meanings.
With the gerund complement remember means recall (a prior eventuality):
I remember visiting the Church of the Holy Sepulcher in 1997. = I remember that I visited the Church of the Holy Sepulcher in 1997.
This means that remember with the gerund complement always has to some extent a retrospective (backward-looking) sense: the act remembered occurs before the moment of remembering.
With the infinitive complement remember means recall and fulfill an obligation:
I remembered to visit the Church of the Holy Sepulcher as you advised. = I remembered your advice and visited the Church of the Holy Sepulcher.
This means that remember with the infinitive always has to some extent a ‘prospective’ (forward-looking) sense: you remember that something must be done and then you do it. Although the verb remember entails your actually performing the act remembered, that act occurs after the moment of remembering.
Now let's look at the test question:
Do you remember ______ the Church of the Holy Sepulcher? (use a form of visit)
The question is cast in the present tense, and it addresses what is almost certainly a unique event.
If the visit were a future event we would be dealing with prospective remember = ‘recall and fulfil an obligation’. But the fulfilment requirement has not yet been met, so there is a contradiction. Consequently, you almost have to be dealing with retrospective remember = ‘recall a prior event’, and the gerund complement is called for, as you guessed.
There are however two circumstances in which a present-tense-form remember might be used with the infinitive.
One is when remember is an imperative: “Remember to visit the Church of the Holy Sepulcher!” Here both remembering and fulfilling the act lie in the future, so there is no contradiction. —But that is not in play in your test question.
The other is when the act remembered is not a unique event but a habitual event, one which is performed repeatedly over a time span which includes past, present and future:
Do you remember to brush your teeth every night?
Here again there is no contradiction because the present-tense-form remember is not tied to a specific time.
Thus, the infinitive complement would be acceptable if the Church of the Holy Sepulcher were your parish church and the questioner (probably your mother!) were asking whether you attend regularly:
Do you remember to visit the Church of the Holy Sepulcher every week?
With a past-tense head clause (Did you remember) the ‘rules’ are the same, but the application is slightly different. With the gerund, the sense is the same as in the present tense, but ‘backshifted’ to an earlier occasion:
Did you remember visiting the Church? = Did you on the occasion we are speaking of remember (=’recall’) that you had visited the Church on some prior occasion?
With the infinitive, however, the sense is more ambiguous:
Did you remember to visit the Church? might represent either
- Did you on the specific occasion we are speaking of remember (=‘recall and fulfill’) to visit the Church? or
- Did you during the timespan we are speaking of habitually remember (=‘recall and fulfill’) to visit the Church?
The ambiguity will be resolved by the context in which the sentence is spoken