You are quite right. A technical correction, however:
In all of these, the annoyance is felt at 'speech time' (ST). The act (or failure to act) which causes the annoyance occurs at 'event time' (ET).
You could look more cheerful. - Hermione is annoyed now (ST) because Harry does not now (ET) look more cheerful.
You might have told me! - The speaker is annoyed now (ST) because the person she is speaking to did not then (ET) tell her what she needed to know.
'Reference time' (RT) does not come into play here, because ET = RT. RT is invoked only when a relative time is expressed - in effect, only when a perfect aspect is involved.
And note that with these irrealis (unreal) modals in past form with present reference, it is impossible to express perfect aspect, because these modals have no participles. In fact, if we want to backshift these utterances, we have to carry them unchanged into the past:
Hermione was annoyed. She felt that Harry might look more cheerful.
I was annoyed. I felt that you might have told me!
When MODALPAST has ST reference, the construction MODALPAST + have + VERBPAST PARTICIPLE, although it employs a 'perfect' construction, does not express perfect aspect but only 'anteriority', pastness.
This is just one more case where the evolving use of modals has thrown a monkey wrench* into the grammatical works.
* BrE = 'spanner'