In this sentence:
He doesn't like young women or any women for that matter
What's the main "matter", his not liking all kinds of women, or merely young women?
In this sentence:
He doesn't like young women or any women for that matter
What's the main "matter", his not liking all kinds of women, or merely young women?
Out of context, I would read that to mean that for some reason, the question of his not liking young women (or whether he did) had come up, and the sentence indicates that he does not like them. Then it expands on the point, to indicate that he does not like women in general.
However, which is the main matter can not be told from this snippet. The comment on women in general could just be a digression while the discussion concentrated on young women, or it could shift the discussion to his misogyny.
The main point would actually be that he does not like women, at all. Without seeing the context around the sentence, the author seems to be using this method of writing to surprise the reader - ultimately letting you know that the man does not like any woman.