As Andrew mentioned, there's lots of ways to handle this, but one thing to keep in mind is the slight nuances in phrasing between some options.
If you just really don't recognize the person at all and are wondering how they know you, you might phrase your response as a question, such as
I'm sorry, have we met before?
This is a fairly polite way of saying you don't think you know the person since you are at least starting with an apology, even if you're not really sure you should be apologizing for anything. That probably would have been fine in the case you describe.
But sometimes you recognize the face but can't remember the name, or you're in a situation where even though you don't recognize the person you're pretty sure they might be someone important that you should know and don't want to be rude to. In those cases you can be extra polite:
Hey, how's it going? Remind me, when did we last meet?
This way, if you don't ask when you've met but use the imperative (in a polite voice) to have them tell you, you're pretending to remember that you did meet before but just saying you forgot what you were doing then.
Another tactic is to not even give away that you don't recognize the person at all. It's not at all uncommon to never address someone by name in conversation, even if they address you by name first, so you could just play along.
Hey! How have you been?
This works well if you're just passing on the street or something where you probably won't talk long and probably won't see the person again any time soon. And if you're lucky, you'll remember who they are by what they start talking about. But if you're at a conference or even just at the bar – some situation where you might run into them again later and have them start introducing you to people – this strategy could backfire on you when someone else asks "So how do you two know each other?"