"You did so!" sounds natural in the context of a childish argument:
You tattled!
I did not!
You did so!
In the context of the original post's example, neither "You did" nor "You did so" is natural. "You did so" is less natural. "You did" is incorrect because "did" is the past tense of "do", not "be".
The first sentence of the original post's example can be changed to use a subjunctive form of "do". (One possibility is to use "do him" in an informal way that means "have sex with him".) With that change, "And you did" would be an equally informal, natural, and grammatically correct follow-up.