Etymology might help:
c.1400, "innocence, blamelessness; chastity, purity," from Old French integrité or directly from Latin integritatem (nominative integritas) "soundness, wholeness, blamelessness," from integer "whole" (see integer). Sense of "wholeness, perfect condition" is mid-15c.
"a whole number" (opposed to fraction), 1570s, from Latin integer (adj.) "whole, complete," figuratively, "untainted, upright," literally "untouched," from in- "not" (see in- (1)) + root of tangere "to touch" (see tangent (adj.)). The word was used earlier in English as an adjective meaning "whole, entire" (c.1500).
late 15c., "of or pertaining to a whole," from Middle French intégral (14c.), from Medieval Latin integralis "forming a whole," from Latin integer "whole" (see integer). Related: Integrally. As a noun, 1610s, from the adjective
The central concept appears related to originalism e.g. the idea of golden age past where everything and everyone was perfect. Integer, integral and integrity all mean complete, fixed and unchanging.
As a practical matter of modern language, someone with "integrity" is someone whose behavior is governed by decisions made long ago that are unaffected by current circumstances. E.g. deciding when one is 20 that accepting bribes is wrong and then, over the course of the next decades, always turning down bribes no matter how large because of that decision made decades ago. That decision becomes "integral" to the persons behavior.
You can see the same sense in the phrase "structural integrity" as applied to inanimate objects. It's means a structure that, because of its internal construction, will not break or collapse in response to changing environmental conditions. Conversely, something without "structural integrity" will only remain standing or functioning in a very limited set of envirionmental conditions e.g. a house of cards.
We like integrity in humans because it makes their behavior predictable. Once we understand the individual's decision-making basis, usually a moral code, then we can predict their behavior in any future events because they will always behave the same way in the same circumstances because the decision has in effect already been made long ago.
We like predictablity so much that sometimes we even use the phrase "perverse integrity" to mean someone who consistenly follows a moral code we disagree withit, e.g. a traditional Mafia member who goes to prison for decades rather than betray the code of Omerta.