Go to is a normal, neutral phrase: it can be used in any tense, and in most tenses it does not imply that the person has or hasn't returned:
I am going to school.
She went to France
They may go to the cinema.
But in perfect constructions (in any tense), gone to usually implies that the person hasn't returned:
She's gone to school. (implies she is still there)
They had gone to France. (implies they had not, at that time, come back)
I may have gone to the cinema when you arrive. (implies I will still be there).
If you want the sense of "gone and come back" in a perfect construction, then you usually need to use the idiom have been.
She's been to school. (implies she has come back)
They've been to France. (implies they have come back)
I may have been to the cinema by the time you arrive. (implies that if I do I will return before you arrive).
You cannot use this idiom (*be to) in any non-perfect tense:
*I will be to France.
*She was to school.
*I may be to the cinema.
are all ungrammatical (marked by the *)
So in your examples, been would be possible, and to my ear more natural. Gone is possible, though, because they are not talking about specific occasions, but over a period of time, and whether or not you came back is not relevant.