To 'knock someone's socks off' means to impress them greatly. It's clearly figurative, as you couldn't literally 'knock' a sock off.
A lot of humour is subjective and contextual, so it's difficult to say exactly what the effect of adding this word is. On one hand, adding a descriptive adjective to describe the word 'sock' could complicate the idiom as the hearer is left wondering if you are referring to their literal socks. Do you think that their literal socks are 'silly'? Has the idiom now become literal?
On the other hand, the word 'silly' can be used as a very mild alternative to an expletive. While certain expletives / swear words are also technically adjectives/adverbs, they only serve to add emphasis to what is being said. Recognised as such, your sentence makes sense and the idiom is preserved. It might say something about the speaker, that they would choose such a childish word to curse with, and some humour could lie in that.