He decided to quickly leave the house.
I believe it was the split infinitive that was the concern. I was taking a verbal aptitude test, and the correct answer was "He decided to leave the house quickly". I suppose that it was because of the split infinitive that this sentence is prescriptively bad.