This use of hedge- as a prefix to the participle stealing may be intended by Kipling to convey the sense that the object of this particular stealing is of a very paltry sort.
OED has this as one of the definitions of hedge- as a prefix:
a. Born, brought up, habitually sleeping, sheltering, or plying their trade under hedges, or by the road-side (and hence used generally as an attribute expressing contempt), as hedge-bantling, hedge-brat, hedge-chaplain, hedge-curate, hedge-doctor, hedge-lawyer, hedge-parson, hedge-player, hedge-poet, hedge-wench, hedge-whore, etc.
The implication would have been more obvious to a reader in Kipling's time than it is today, and certainly obvious to a speaker in the era of the Napoleonic Wars (1803-1815) in which the story is set.
It is also possible, as you suggest, that he refers here to the actual stealing of hedges for use as firewood, an act which was specifically sanctioned in English law, and noted in some accounts as a problem during that era, and which would be expected only of an impoverished person, which Master Jerry is at pains to make clear he is not.