In English, the second and third statements are correct. "I'm going" indicates a future event and the after indicates a sequencing.
If you are talking from the time where the finishing and walking have both not occurred yet:
I'm going for a walk with him after he finished his work
This sentence would be incorrect because the finishing is in the past, but the walking is in the future, so the sequencing of "after" doesn't work.
I went for a walk with him after he finished his work
This works since the times are in sequence
If you are talking from the time where the finishing is completed but the walking has not occurred yet:
You will have to join them with "since" or "because".
I'm going for a walk with him since he finished his work
I'm going for a walk with him since he has finished his work
These work because "since" (or "because") indicate a condition or state reached. Since has another meaning which is similar to "after", but that is not applicable in this case.
Since he arrived, I've been rather busy
From the time he arrived, I've been busy.
The difference between the second and third would be slight. The present perfect would imply a relevance between the completion of the work and the present action (going for a walk). Either is possible, but in my dialect at least, the present perfect would be more common for processes (like eating, working, etc).