"As for where you're going to" is the way to say this, though as the commenters suggest, it would be common(er?) to drop the "to".

As you suspect, "As to where you're going" is not begging for that same "to" at the end.

If you replace "where" with a noun phrase you'll get a much greater demand for that "to":

As for the house you're going to...

Either way, "as for to" is so strange that it led me and perhaps one commenter to misparse your sentence and rule it un-English. A closer look reveals it to be theoretically acceptable ("As for the question of to where you're going"), but wholly unnatural.

"As for where you're going to" is the way to say this, though as the commenters suggest, it would be common(er?) to drop the "to".

As you suspect, "As to where you're going" is not begging for that same "to" at the end.

If you replace "where" with a noun phrase you'll get a much greater demand for that "to":

As for the house you're going to...

Either way, "as for to" is so strange that it led me and perhaps one commenter to misparse your sentence and rule it un-English. A closer look reveals it to be theoretically acceptable ("As for the question of to which house you're going"), but very improbable.