In the dictionary
let somebody/something go | let go (of somebody/something): to stop holding somebody/something
Don't let the rope go.
Don't let go of the rope.
Let go! You're hurting me!
I think we often say "let go of a thing" when that thing is leaving us or moving away from us or trying to escape us.
For example, 2 children are fighting over a car, and they are pulling the car towards themselves. And you say, "let go of the car" (the car is moving away from you).
Now, your son is holding tight onto a chair. The chair stands still, it won't move away from the boy.
Can we say "let go of the chair" when the chair stands still and won't move away from the boy?