Either could be correct, depending on what happened, or what the speaker wants to focus on.
With the simple past, it means: first, Mrs. May went close to the hedgehog and stopped, and then because she had moved closer, she noticed the yogourt pot.
With the past continuous, it means: while Mrs. May was moving towards the hedgehog, she noticed the yogourt pot. The context suggests there's a connection between moving closer and noticing the pot, but the grammar does not. Also, this version of the sentence doesn't indicate whether Mrs. May stopped moving.
In terms of naturalness, the simple past sentence wins easily for several reasons:
- With continuous actions, while it's correct to use "when", it's much more common to use "while"
- Because Mrs. May is consistently moving towards the hedgehog, "closer" makes more sense than "close"
- In this context with the continuous, the "go close(r)" is unnatural. "Move towards" is much better.
When Mrs May was going close to the hedgehog, she noticed that there was a plastic yoghurt pot on the hedgehog's head.