- The captives were taken away on his order(s).
- The captives were taken away under his order(s).
Dictionaries are vague about the difference between "on his order" and "under his order" and whether "order" should be singular or plural.
On his order (or orders) is an adverbial used to indicate either
the authority by which a specific action is carried out—this may use either the singular order or the plural.
Flags were lowered on the president's order.
the signal on which an action is to commence—here order is singular.
"Fire on my order ... Fire!"
Under his orders may be used in the same way as #1:
The Colosseum was built under the orders of Emperor Vespasian.
But it also acts as
The Fourth Division was placed under the orders of the general officer commanding the Second Army Corps.
Circe now came under the orders of Commodore Sir Home Popham.