It seems that there is a story behind the sentence "you can always tell a Harvard man" and since I do not know the story, I do not understand why and where people use it.
It's a joke. The entire line is:
You can always tell a Harvard man, but you can't tell him much.
It uses two different senses of the word 'tell'.
You can always tell a Harvard man
means you can always identify whether someone is a Harvard man (graduate of Harvard) - because he is sophisticated, smart, wealthy, etc.
but you can't tell him much.
means that you can't speak to him with new information or your opinions - because the stereotype of a Harvard graduate is conceited, arrogant, and pompous.
The two definitions of tell (MW) are:
Transitive verb, definition 6
to find out by observing, recognize "you can tell it's a masterpiece"
Transitive verb, definition 4a
to give information to, inform
"you can always tell a Harvard man" please add, "but you can't tell him much."
Presumably coined by a Yale man/student or graduate, it makes reference to the University of Harvard, an Ivy League school (one of a few highest/best education/most expensive and prestigious universities, like Yale) in the USA. The 'Yale' man would have said this as a put down or joke.
ON edit: The Harvard man or Yale man are current or former students or graduates of Harvard University or Yale University.
The Harvard man would dress, speak and be educated to high standards. He would be noticeable by the way he carried himself -- his dress, speech patterns, job he had, car he drove, firm he was employed by. So the Yale man in recognising that he is similar, notes that the Harvard man is so self-assured and thinks so highly of himself, that he is incapable of learning anything new. Unlike (it might be supposed) the Yale man. He cannot be told anything.