This is an sentence in my english essay. What I want to say here is that I spent some time on math exam. And the time I spent on it is not that much, however is not short either. But "not a short period" looks strange to me in the sentence since I never saw anyone using this before.
Your sentence is a example of litotes. It is rather subtle and depends a lot on context (and in speech on tone or body language). The sense is "ironic understatement" and the purpose is to emphasise the point.
The classic example of litotes is saying something is "not bad". The can mean anything from "satisfactory" to "excellent" depending on context and tone.
So if you say "I spend not a short time preparing for my maths exam." That might mean "longer than expected".
You can avoid the litotes and say "I spent quite a long time". That is not very long, but not short either. Or "I spent enough time" (different meaning but may be acceptable). Or "I spent some time" (rather vague).
"I spent not a short period" could never be correct in that context.
If you want go into tiny detail more relevant elsewhere, "no short period of time" might be acceptable…
Almost separately "I spent (any description of time) for the math exam" is not correct.
Your idea requires an auxiliary, as for instance "I spent hours preparing…" or "… days revising" or "… weeks studying for the exam"