... There was a rush to buy the cook drinks and hear more details(about Frank).
"Always thought he was odd," she told the eagerly listening villagers, after her fourth sherry. "Unfriendly, like. I'm sure if I've offered him a cuppa once, I've offered it a hundred times. Never wanted to mix, he didn't." ...
The sentence seems to be elliptical. I think it could mean either: "I never wanted to mix(get along) with him, because he didn't want to." or "He really didn't want to mix with anyone else." What does it truly mean?
Because of the previous first person perspective within the quotation, one might be tempted to think, as you state, "I never wanted to mix (get along) with him, because he didn't want to." But her references to frequent offers of tea or coffee would point more to Frank's lack of interest in "mixing."