I have heard some of my townsmen say, "I should like to have them order me out to help put down an insurrection of the slaves, or to march to Mexico, - see if I would go"; these very men have each, directly by their allegiance, and so indirectly, at least, by their money, furnished a substitute.
You understand the verb-phrase, but in this specific context, the indirect object is not stated:
to furnish [someone] with a substitute
That is, to furnish [the government] with a substitute, which here means to send to the government someone else to fight in one's stead.
It was common practice for conscripts to pay a certain amount to avoid having to serve in the army themselves.
... these very men have each, directly by their allegiance, and so indirectly, at least, by their money, furnished a substitute.
Let's reorder the clauses putting the adverbial phrases at the end:
These very men have each ... furnished a substitute directly by their allegiance and [done the same] indirectly by their money.
so can be expanded to done the same.
The phrase "their money" need not refer literally to paying such a fee to avoid conscription, but to taxes paid. The argument advanced in that paragraph of the essay (your excerpt contains only a few words of it) is that the man who refuses to serve in what he considers an unjust war must have "clean hands". The man who would refuse to serve must end all allegiance with those furthering the unjust cause. He cannot merely speak out against it and at the same time be paying for it indirectly. His paying taxes to support an unjust war is tantamount to his sending a substitute.
P.S. I tried to take a shortcut above when talking about the implicit recipient in the furnish "transaction" so to speak. Consider the following.
We're having a party on Friday.
-- Great! I will furnish the beer.
In that case, the beer is what I will bring to the party. There is no mention of the beneficiary or recipient.
The boosters club is going to paint the gym on Friday. The school is going to furnish us with paint.
In that case, paint is what is being furnished (supplied), and us is the recipient / beneficiary. Our status as beneficiary is expressed by the objective case, us not we.
In the Thoreau essay, those men, by paying money into the government's coffers, are furnishing a substitute. There is no explicit mention of the recipient or beneficiary of the action. But it is understood from the tenor of the passage (and from the next paragraph) that the beneficiary is the government.
