As @KateBunting said, there is no significant difference. They are two valid ways to say exactly the same thing.
As you can see in other answers, many English speakers will perceive a slight distinction, however this is really a matter of subjective (and unreliable) interpretation by the listener, and not something learners really need to concern themselves with.
If you really want to get 'into the weeds', my feeling is that although the second sentence is not in the passive voice, the use of the present continuous makes it feel like more of an indirect inquiry. This might be perceived as softening the tone of the question.
But consider this alternative:
Person A: I'll call you tomorrow and we'll talk about the trip.
Person B: Alright. What time?
This is much more likely as genuine modern spoken English, but is far more 'direct' than either of the alternatives. But nobody would ever consider it to be 'impolite'. It's simply asking for the time succinctly. There's really no need to over-complicate the situation.