...or should I say "yesterday mathematics teacher"?
Usage in a full sentence:
'I'm waiting for yesterday's Mathematics' teacher to come online'
American here, so your usage might vary by country. I think it will be dependent on your intention.
Yesterday, the mathematics teacher taught algebra. Yesterday's mathematics teacher taught algebra.
I think both are fine but mean different things. The first one means that the normal math teacher taught algebra yesterday. The second infers that there was a different teacher (substitute teacher possibly) yesterday that taught algebra.
In your example, I think this would be what you want to do:
I'm waiting for yesterday's mathematics teacher to come online.
The teacher who taught mathematics yesterday is "yesterday's mathematics teacher". The teacher who (now) teaches philosophy is the (current) philosophy teacher.
While a teacher may personify his discipline and say that he belongs to it, it would be unconventional. ;)