The differences are a bit difficult to explain.
For example, the way I would personally respond would have been:
"Are you sure you did not see her?"
"Yeah, I'm sure. I'd remember if I had seen her."
"Yeah, I'm sure. I'd know if I had seen her."
and the variants go on ... "Had I seen her, I'd remember."
In this specific context of the boss asking, I'd probably say something like "Yeah, I'm sure I haven't seen her. Had I, I would tell you." (or "Had I've, I would've told you") Which implies that there's nothing wrong with my memory (because why would/should there be) and points out/reconfirms that had I actually in fact seen the secretary, I would have told the boss the first time he/she asked.
(If the boss is a native speaker: Native speakers tend to do this double confirmation question thing when they're surprised (especially when looking for something/someone)... "Are you sure you haven't seen XYZ?!" Usually it's not meant literally (as in questioning the other person's memory or questioning the other person's honesty) ... it could, but quite often it's just an expression that expresses (1) an element of surprise/frustration that you can't find XYZ and (2) expresses that you really want to find XYZ. (As opposed to asking once in passing - which could indicate a lower level of importance)