If you use 'a new year', you are referring any new (current, not past) year in it's entirety. 'A new year' is the entire new year from January through December.
Using 'The new year' can have the same meaning as 'a new year', but more often means the beginning part of a new year. The time is not specific but may mean anything from the first couple days to the first couple months of the new year.
'Any new year' is more general and refers to every year when that year was new.
The idiom "turning over a new leaf" relies on that specific set of words. Replacing 'turning over' with 'turnover', or replacing 'a' with 'for' will result in a sentence different enough from this idiom that it will probably confuse your readers/listeners.
I recommend you use the sentence 'The new year means the turning over of a new leaf'.