As has been pointed out by others, the "logical" argument for I didn't use to do that is that did already carries the verb tense, so it's not needed in used. Same as, for example, I didn't have to do that.
But as this NGram shows, usage has changed considerably over the past century and more...
Unsurprisingly therefore, you'll find plenty of traditionalist grammarians defending the older style (which is completely inaudible in normal speech anyway). But it's pretty obvious which version has the upper hand today, so I suggest you go with that unless you want to look like an old fuddy-duddy.
As pointed out by John Lawler in an ELU answer on this topic...an ELU answer on this topic...
Both [spellings] look bad, the first because used looks like a misspelled infinitive,and the second because use to doesn't look like it sounds like used to should.
This should be considered a bug in the orthographic system.
It's worth noting parallels between the voiced and unvoiced versions of used to / have to. In both cases the unvoiced version has a totally different meaning (relating to habitual action / obligation, rather than employment / ownership). If you'd never seen written English, you'd naturally classify the voiced/unvoiced versions as different words, not different pronunciations of "the same" words.
Interestingly, the "special" version of have to actually does have two different pronunciations itself (past tense I had to go then is pronounced hat; in present tense I have to go now it's haff). There's no such split with used to because in semantic terms it's always "past tense" anyway. And since language use is primarily driven by spoken rather than written forms, this is essentially why people increasingly tend to ignore the pedagogic / logical arguments and stick with used to in all contexts.
