He stopped to talk means that he interrupted whatever he was doing at the time and began to talk. The infinitive expresses the purpose.
He stopped talking on the other hand, has the opposite meaning. It means he didn't talk any more, he was silent.
He went on to talk (about) This means the speaker changed the topic of discussion, and started a new one.
He went on talking (about) here the speaker continued the action of talking, this might have included one or more interruptions however, the topic remained the same.
He continued talking There are two possible interpretations; in the first, he talked virtually uninterrupted. In the second, despite being interrupted he kept on talking
He continued to talk This means he talked continuously, and probably without any interruptions.
In other words, continue is one of those verbs that can be used with the infinitive or the gerund without any changing in meaning.
Verbs that take gerunds or infinitives as verbal direct objects (as listed by Grammaring)
BEGIN, START, CONTINUE, CEASE, DREAD, INTEND, LOVE
Go on is defined by the Free dictionary as To keep on doing (something): "Don't go on talking."
Continue TFD v.t. to go on with or persist in: "to continue reading".