Some 40 percent of people's "first memory" happened at an age when it wasn't possible to create memories.
Context shows the percentage is about the people. The word memory, being singular, also indicates the percentage is about the people. Without context, would you be able to infer which is meant if we changed the sentence to:
Some 40 percent of people's first memories ...
And what about this one:
20 percent of the countries' money came from tourism.
Is there any rule for the structure "n % of X's Y"?