I have googled and rarely found the example of usage. Instead I found "for the purpose of" is more common. My English teacher, who is not English-native speaker, showed me one but I rather disagree with her. For example: The spy dyed his hair (,since/,so/;hence,/for the purpose that) he wouldn't be recognized. Her answer is "for the purpose that". I thought "so" or "hence" is more suitable.
So, hence, and for the purpose that are all correct. It depends on your context and where you are speaking.
The "purpose of" something is the reason that it exists, its intended function. This is an abstraction, typically.
The "purpose for" something is an actual use. This is a concrete example.
The "purpose that" something is used when there are multiple purposes and this is calling out one in particular.
The spy dyed his hair for the purpose of blending in to his surroundings.
The purpose for the spy dying his hair was to blend in to his surroundings.
The spy dyed his hair for the purpose that he was meant to deceive the enemy.
These can interchange a bit with subtle meaning shifts.