I'm a native English speaker. My immediate interpretation was that the intended meaning was "so it is with me", and the alternative "as for me" doesn't sit right.
For a definitive answer, we'd have to ask the translator of that particular text, Katherine Woods, and this is complicated by the fact that she died in 1968. (Some have commented and/or answered that we ought to look to the original French text for the answer, but I think this is misguided. A good literary translation shouldn't simply be a word-for-word translation of the original text.)
We can make some guesses. Woods' translation is widely acknowledged to be the most "poetic" available translation despite (or perhaps because of?) the fact that it is not as word-for-word accurate as other translations. While it's admittedly a subjective assessment, the use of the phrase "so with me" to mean "so it is with me" seems significantly more poetic and a better match for the overall style of the translation than imagining she meant it in the other sense.
Also, "So with me: ..." punctuated with a colon also seems like a rather awkward way of writing "as for me", or "in my case". If this was what was actually meant, wouldn't it have made more sense to punctuate it "So, with me, ..." or "So with me, ..."? And wouldn't "So, as for me, ..." or even just "As for me..." be a more natural way to phrase it?
Note that Woods uses the much more natural construction "As for me, ..." in three places:
- As for me, I was upset over that bolt. [Chapter 7]
- As for me, I am concerned with matters of consequence. [Chapter 13, the same chapter your quotation comes from.]
- "As for me," said the little prince to himself, "If I had fifty-three minutes to spend as I liked, I should walk at my leisure toward a spring of fresh water." [Chapter 23]
Unfortunately, "So for me" isn't used anywhere else, so we can't compare other usages. However, I think it's significant that it's punctuated with a colon here. The text I'm referencing reserves colons for two purposes, as far as I can see: (1) to introduce a quotation or a figure (e.g., "My Drawing Number Two looked like this: [drawing]); and (2) to join two related but independent sentences. It is never (mis)used as a substitute comma.
Now, I can't say for sure whether Woods or some editor along the way decided on this punctuation, but I think this provides some support for the idea that "So for me" was intended as a standalone sentence, and this supports the "So it is for me" interpretation.
So, to sum up, on the basis of:
- the poetic style
- the choice not to use the more natural construct "As for me, ..."
- the punctuation
I think the "so it is with me" interpretation is more likely correct.