Native speakers wouldn't find the original sequence from NYT at all "confusing".
It's perfectly familiar (if somewhat stylised) phrasing, where the noun phrase construction not so much X as Y carries the highly specific implication expected = X, reality = Y.
OP's first attempt at "improvement" (They didn’t rent a home so much as a fraction of one) doesn't work so well as the original because it erroneously links negating n't = not closer to the verb (rent) rather than its true referent (a home). That "misplaced" negation weakens the amusing/unexpected contrast intended by the writer, because it puts the "focus" in the wrong place.
To my mind, the second rephrasing (They rented not a home but rather a fraction of one) also falls short of the mark because it lacks any strong allusion to that "expectation vs reality" clash. You can tell this by considering something like I drive not a diesel but rather a petrol car. Admittedly, that's rather affected phrasing, but the point is this construction can be used without implying that the speaker might have been expected to drive a diesel (feasibly the speaker only mentioned diesels at at simply because they'd been referred to earlier in the conversation).
TL;DR: There's nothing syntactically "wrong" with either of OP's alternatives, but for a competent native speaker they're not so effective as the original. And it may be worth pointing out that NYT's target readership is competent native speakers, not people who are learning English in later life.