I believe the author is trying to say that the main character thought (without letting it be known to others, i.e., to himself) the following:

"Unless you try to talk to guys, genius," he thought.

I believe the author is trying to say that the main character thought (without letting it be known to others, i.e., to herself) the following: “Unless you try to talk to guys, genius.”

This could be made clearer by writing it with quotation marks, like so:

“Unless you try to talk to guys, genius,” I thought to myself.