"I should rather say a very particular friend of your son's."
Not "of your son"? Why should I use possessive case in this statement?
This is an idiom. It does not really make much sense. Some people call it the "double possessive." We also say "A friend of mine" instead of "A friend of me," "A friend of yours" instead of "A friend of you," and so on. In your original example the authorities do not allow "of your son" without the apostrophe-s (although it sounds natural to me either way); and in my examples it would be obviously incorrect to use "me" instead of "mine," etc. Where a pronoun is in that position in the expression (the object of "of"), it must be possessive.