I know the phrase "I know (someone) in person/personally." However, I didn't find the word "directly" used with that phrase. Also, I don't find that "in person" and "personally" fit the situation I intend.
Note: X, Y, and Z are all females, so I'm going to refer to them individually by "her".
X told Y about Z some basic and general information as Z's name and workplace. Now Y wants to tell her mother that she didn't meet, see, or talk with Z. So, Y says: "I don't know Z directly, but I know her from X."
In person means involving someone's physical presence rather than communicating by phone, e-mail..etc
Personally means if you meet or know someone personally, you meet or know them in real life, rather than knowing about them or knowing their work.
If "directly" isn't proper, I may go with "personally".