Can we use
A perfect start to a day
A perfect start of the day
Please explain me properly
Grammatically, "a perfect start of the day" makes sense. And without the word "perfect", it would usually be the best choice, as in "at the start of the day".
But there is an established idiom for phrases like "a good start to" when describing some period of time or event, where the adjective good can be replaced with many other adjectives describing value quality: great, perfect, excellent, bad, poor, disappointing, etc.
Since probably most English speakers are used to this idiom, it will be better to say "a perfect start to the day".
(Interestingly, it looks a century or more ago, "a good start of" was more common than "a good start to", but "a good start to" has been steadily becoming more frequent: see the Google ngram comparison.)