If your back faces someone while you are sleeping, are you sleeping with your back "at them" or "to them"? Example: "She slept with her back AT him" OR "She slept with her back TO him"
TO is correct. You can get a sense of this from Google n-grams.
X to Y has a lot of meanings and one of those is a destination or direction identifier - it's used to express the target/direction that X is pointing or facing.
X at Y answers the question where is X? and this typically works in English only if Y is a place or a point on some type of line, sequence, or continuum. This concept is usually invalid for a person's back unless the question where? is involved, implied, or identified as a concern for the speaker - e.g. "My hands are at her back".