of you refers to "you" (him, her, them, etc) as possessing a quality or attribute.
You have behaved in a way that is characterized by an adjective and that attribute is then attributed to you (her, him, etc).
It was generous of you to lend me your car so I could pick up my girlfriend at the airport.
The (possibly optional) complement specifies the action or behavior in the form of a non-finite clause:
It is kind of you to say so.
The non-finite clause supplies the idea-content for it exposed at the head of the sentence.
It (i.e. "to say so") was kind of you.
If the thing to which it refers is already established in context, the non-finite clause is not needed:
I didn't mention your prison record when introducing you to my parents.
-- I know. I was there. It was kind of you.
But we will have to let them know at some point.
With the noun, we would say on your part instead of of you.
It was a kindness on your part not to mention my juvenile prison record when introducing me to your parents.