Alright, first off a quick summary of the two verb tenses you're asking about.

> **Past Perfect** [*had verbed*] - This indicates that something happened further in the past than the time I'm referring to. If I was asked "*Was that night the first time you drove a car?*" I would answer, "*No, I **had driven** once before that." The event I'm referring to (my first time driving) was already in the past when the time already in question (that night) occurred. Past perfect is the correct verb tense in this context.

> **Past** [*verbed*] - The simple past tense just indicates that something happened before the present. It is used when I am referring to a single event in the past or to multiple **concurrent** events. For example, "*I **went** to Europe*" or "*I **was** in Belgium when they won the World Cup.*"

I do agree with StoneyB's comment that the question/response you've listed sort of lacks a logical relationship without a story around it. I also agree with FumbleFingers' comment that (again, without context) being out of town "until that moment" seems odd. So, the correct verb tense really depends on what you're trying to say. Presumably it's that you were not in town when this event happened last time. In that case...

How many events in the past are we referring to? Just one. The "last time" this thing happened, during which you happened to be out of town. Therefore the **simple past tense** is the correct one to use.

> "It couldn't have happened, because I was out of town [**at**] that moment."

If my assumptions to what you were trying to say are incorrect please just edit the question or post a comment to explain that and I'll be happy to edit my answer.