The "casual, lighthearted" intent is different from the formal use of conditionals and the "were subjunctive" (and as @JavaLatte says "anastrophe").
Moreover, telling someone "I'm not going to do something in return because you are a friend." seems odd to me. The proper attitude should be "I'd do the same for you."
Perhaps this is cultural. It is grammatically correct to say
I'd owe you one if you weren't my friend.
In a culture in which this is can be understood you can remove the conclusion (since your friend understands the context)
Thanks! If you weren't my friend...
But in my culture I think I'd be more likely to say
Thanks! You're a great friend. I owe you one.
You indicate that this is intended ironically: Irony is hard in a foreign language. It is, perhaps would be more clear if the "debt" was ironically small:
I'm so glad it was you that did this for me.
Why's that now?
'Cause if it'd been anyone else, I'd feel I like I'd have to buy them a pint to say thanks, and I'm totally skint!
There are lots of "flags" here that the speaker is not serious: the very casual style "'cause". The small offer of repayment "a pint". The exaggeration "totally skint".
This kind of dialogue is more "natural", but it can't be prepared. You can't learn it from a book.