I understand that after the verb MAKE, the verb that follows should be written without TO, Example: "They made him pay all the money" however, why it sound so correct to me: "He was made to pay" Then again: "They made me pay". Is it the presence of the object pronoun that makes the difference?
In English, when we combine two verbs usually the second verb needs to be in either the infinitive or gerund form. The situation you are asking about is one of the cases where the infinitive form is used for the second verb.
This website has a decent list of verbs that should be followed by infinitive verbs, but it is not entirely complete: https://grammar.yourdictionary.com/parts-of-speech/verbs/infinitive-verb.html
Knowing when to use infinitives vs when to use gerunds or regular forms of verbs can be confusing, but it's something you get a feel for over time.