I know that this question can be applied to other words in their adjective and adverb forms. But as I came across "extreme", I'm specifying my question about it.
This extreme/ly bad habit of yours is so annoying.
Don't remind me of that extreme/ly hard experience.
I learned that adverbs come after verbs to describe them, but the above sentences don't describe their verbs, and as "extremely" means "very", it seems reasonable to come as an adverb since "very" would fit well. However, it would be logical for them to function as adjectives since they describe the nouns that follow them.