"Extra hours" is a coherent phrase. "Extra" directly modifies "hours". "Extra" is an adjective, and "hours" is a noun.
Either "to sleep" is transitive, which allows "extra hours" to act as the infinitive's direct object, or "to sleep" is intransitive, which requires "extra hours" to act as an adverbial.
In this case, I find the adverbial explanation more sensible. "Hours" is a temporal noun, and temporal noun phrases often act as adverbial constructions. "To sleep extra hours" carries the same meaning as "to sleep for extra hours" or "to sleep through extra hours". Wikipedia's entry on adverbials explicitly mentions temporal noun phrases as a common adverbial form.
The extra hours in question do receive any action. They merely represent the duration of the action.
In this sentence, "extra hours" is an adverbial noun phrase which contains the adjective "extra".