My father gave me \ has given me my name.
Well the action "giving my name" happened and finished in the past, but my name has never been changed since that time so the action still affects on the current situation.
Which tense should I use?
"My father gave me my name" is better for exactly the reason you say - your father completed this action more or less instantly, in the past. A native speaker would not ever use "has given me" in normal speech, in this context. You can get convey some impression of the lasting effect, by using the passive "I was given my name by my father..." as this construction slightly shifts the emphasis of the sentence to the action of naming you, rather than the person who did it.