First, I'm going to agree with the comment made by Jeff Zeitlin; I, too, will usually tell the first caller:
I have another call coming in.
But, if you are looking for a word for the caller, one word you could use is party.
The word party, of course, has many different meanings. For example, there are birthday parties, rescue parties, and political parties. But there is another meaning for this word:
party (noun) A person using a telephone
The word party does not, by definition, mean "the second caller". However, you could use the word another as a determiner, and make this work:
Excuse me, there's another party trying to reach me.
I'm not saying that I'd recommend the word party over, say, caller, or even person:
I'm sorry, there's another person calling me; do you mind hanging on for a moment?
In fact, I think party almost sounds a little old-fashioned in the context of telecommunications; it reminds me of a bygone era when people used telephone operators instead of cell phones, when children shared the same phone number and land line as their parents.
Nevertheless, I thought this quaint suggestion might be useful and informative to our community of learners.