They both seem correct to you because you're assuming but and yet are interchangeable synonyms – but that is not always the case!
The difference is similar to the difference between however and nevertheless. Those two words are also interchangeable in many – but not all – instances.
If yet were properly used in the example sentence, it should be rephrasable to this:
Even though Joey realized he had forgotten his wallet, Joey and Micah made their way to the head of the checkout line.
This doesn't explain the situation very well, because Joey and Micah would have made their way to the front of the line regardless of who had remembered or forgotten their money or their wallet.
Let's alter the example to a case where yet is used properly:
Joey and Micah made their way to the head of the checkout line, yet neither of them had any money.
I can rephrase this as:
Even though neither of them had any money, Joey and Micah made their way to the head of the checkout line.
This makes a little bit more sense. If I found this passage, I would assume that the author was going to explain why the two men got in line even though they didn't have any money.
The word yet doesn't make a good coordinating conjuction between the two men getting and waiting in line and Joey's sudden realization that he didn't have his wallet.
It's also worth noting that the corrected version doesn't use but, but but then. The then is a key word that makes the sentence sensible; it indicates when Joey realized his wallet was at home – after they had reached the head of the line. Without that word, neither but nor yet make apt conjunction.
By the way, it's a very subtle difference that I had to think about for some time.