He is trying to coyly say that the loan of a million dollars was a modest loan from his father. Like if a father gave their son a small loan of, say, a thousand dollars to help buy a car.
But to 99% of the American public, a million dollar loan is by no means "small". In fact, the vast majority would never be able to get such a loan. That is why he is being ridiculed. He wants to play the loan off as being something small and modest that anyone could get. It was no big deal. But the public sees it as an example of his being born into a wealthy privileged family, and that wealth and privilege gave him a significant head start to get to where he was today.