I have been searching for a while, and I haven't seen a good discussion of this phrase. It seems to me like the preposition is in question (I've heard it both ways), and the possessive is also in question (I have arguments for both).
First, the possessive. The moment is personified and it is noticing. So the phrase would be sort of magical. You're imagining a moment as a person and that moment would have very brief notice, so the meaning works. That is, almost no notice at all. The problem with this is that it's a bit crazy. How would we know how much notice a moment would give? How did something so poetic become common usage?
The other interpretation (without the possessive) is that it's a measure of the amount of notice. One moment of notice, but then I suppose it should be singular.
What are your thoughts?
Edit: It has been suggested that this is a duplicate of the "24 hours(') notice" problem. I think it's a subtly different problem. 24 hours notice can reliably be considered an amount of notice, so the phrase can certainly be interpreted as correct without the apostrophe. Moments is more complex because it can't be plural (as noted in the original question and as reinforced in the discussion) and therefore, we enter the complex discussion of the possessive's correctness. Personally, I think that discussion was worth having, and I think the answer was worth it, but given the comments (now chat), it seems like my understanding of what could/should happen on a site like this is different from the community's understanding.