A modal verb is a type of auxiliary (helping) verb that has no meaning on its own but modifies the main verb, changes its meaning and gives more detail about the action.
The verb would is a modal verb used to express willingness in the past, polite request, preferences/desires, offers and invitations.
So in this case, you are using 'would' to talk about the actions that took place in the past. You could also omit would for 'used to' or simply the past tense form.
- When people would ask me how old I am, I would answer them correctly. >>> When people used to ask me how old I was, I answered them correctly.