It seems to me that in the context of your conversational exchange, the utterance
"Hi. How are you? Thanks for calling me back!"
was probably anticipating a response along the lines of
"Thanks, I'm fine, and you're welcome. What can I do for you?"
As Malvolio observed earlier, "Good talking to you" customarily comes at the end of a call, where it functions as a phatic closer to the conversation. So I suspect that your interlocutor may have felt that your response violated two unwritten rules of telephone etiquette.
However, nowadays native speakers of English often find themselves talking to non-natives, especially in a business setting, so they are accustomed to making allowances for such verbal lapses.
This one, which is pretty minor, probably isn't worth losing any sleep over -- that is, unless you have other reasons to believe it cost you a multi-million-dollar contract, your job, and the respect of your bosses, your spouse and your dog...
To answer the last part of your question -- I think your contribution to the exchange about seeing each other on Saturday was fine in terms of its tone, despite the fact that it does not stand scrutiny regarding its logic.
This is because it is in effect saying that you too are looking forward to seeing yourself on Saturday.
It therefore has something in common with another formulaic but logically incoherent conversational exchange:
"I love you, darling!"
"Me too!"