Other answers quote examples, but there seems to be examples for both possible answers, so maybe the answer is "it depends...".
But on what?
It seems to me that it depends on the impression that the speaker wants to give. To borrow Varun KN's example, consider the following two sentences.
If I were her cup, she would kiss me every time she took a sip.
If I were her cup, she would kiss it every time she took a sip.
Both of these are, in my opinion, grammatically correct, but the meaning is worlds apart. The first conveys a strong impression of somebody in love: they would even be willing to be a cup, just to be touched by her lips. On the other hand, the second is a prosaic description of a mechanical process- lip meets cup.
Looking at the sentence in the question, we don't really have any background for the reason why the speaker wanted to kiss him, but we know that this is a real situation that happened in the past (would have) and we know that the speaker did not kiss him- because he is male. The appropriate pronoun is therefore him.
If he were a girl, I would have kissed him.
Let's change the scene a little, and imagine that the speaker sees somebody like this
The speaker finds him very attractive: they don't normally go for men, so they really wish that this guy were a girl, because they can imagine kissing those lips. Omit the have to make it an ongoing wish, and we have:
If he were a girl, I would kiss her
The speaker is visualising this attractive man as an equally attractive girl, and they really would like to kiss her, so the appropriate pronoun is her.