'He is able to speak English'
'to speak' is a to-infinitive usage? If so, 'to speak English' modifies 'able' as an adverb?
Or 'be able to' is one chunk unit to learn by heart as an idom?
The verb phase is "to be able (+ infinitive)".
He is able to speak English.
There is no adverb.
The + infinitive part isn't required. "He is able" is a complete sentence. But if you're going to follow up with what he is able to do (infinitive), an infinitive must follow.