The doctor was said to be a king in this island.
The doctor said "I don't want be a king". "I should think, Doctor." said I, "you'd feel rather proud and glad. I wish I had a chance to be a king."
"Oh,I know it sounds grand." said he, pulling on his boots miserably."But the trouble is, you can't take up responsibilities and then just drop them again when you feel like it. I have my own work to do.Scarcely one moment have I had to give to natural history since I landed on this island"
This is from a novel "The voyages of doctor Dolittle".
I can't understand the meaning of this sentence "But the trouble is, you can't take up responsibilities and then just drop them again when you feel like it." Could you teach me? And are I and have in the phrase "one moment have I had to" inversion?