This was the long way, which we had to go.
I think this sentence is grammatically correct and which is referring to the long way and there is no need to use that to restrict the long way because as in the sentence, this already clearly specify which one is indicated by the speaker, so I guess which can work as a non-restrictive pronoun and which is modifying go adverbly as well as referring to the long way.
Is my thinking correct?
The sentence is grammatically correct.
This was the long way, which we had to go.
As written, the clause is non-restrictive or non-defining, because it is separated by a comma and uses the relative pronoun "which."
The clause "which we had to go" is a relative clause, acting as an adjective phrase to the antecedent noun phrase "the long way." The pronoun "which" thus represents "the long way" and is the object of the clause (i.e. "we had to go [the long way]"). It is not correct to say that "which" is an adverb modifying the verb "go."
Because "go" can be transitive or intransitive, and "we had to go" is by itself a rather idiomatic way of saying "we needed to leave" (which is not intended in this sentence), the phrasing is slightly awkward, particularly because the object pronoun does not follow the verb when it heads a relative clause.
It's also awkward phrasing because without an adjective or determiner, the clause doesn't even work (you wouldn't say simply "we had to go the way"). The reader has to do a couple of mental steps to parse the sentence.
If you replace "go" with a verb that is primarily used in a transitive sense, the structure is clearer:
This was the long path, which we had to traverse.
The analysis of this sentence is exactly the same as the first, but to my ear it sounds more natural and less "forced."
It is also valid to interpret the clause as defining or restrictive, although not strictly necessary. "This" refers to a specific path, presumably, but that's not exactly the same as stating that "the long way" was in fact the way we had to go (they could refer to different things, as in "this [here on the map] was the long way [but we took a shorter route]" etc.).
For a restrictive clause, you would remove the comma, and you could use either "that" or "which" as the pronoun (you don't "need" to use "that" even in a restrictive clause).
This was the long way that we had to go.
or
This was the long way which we had to go.
In this case, the relative clause "that we had to go" defines the long way.
There are also a couple of ways to rephrase the sentence entirely to make it more clear.
As mentioned in the comments, the relative pronoun could be the subject of the clause in the passive voice:
** This was the long way, which had to be gone by us.
This sounds even more forced than the first, and it is several steps removed from the idiomatic "we had to go the long way" that the sentence is trying to convey.
You could again replace the verb, which is better, but the passive voice is not the best choice to represent a sentence of action:
This was the long way, which had to be traveled by us.
This removes the confusion over whether "gone" is an adjective or a participle, but it is still awkward.
If I were writing this sentence, I would forgo the relative clause and form an independent clause with a coordinating conjunction:
This was the long way, and we had to go that way.
The repetition of the word "way" may be considered a pleonasm, but the rhetorical emphasis of each clause is different (the first emphasizes the adjective "long," and the second emphasizes the verb "go"). Moreover, the SVO ordering is much more natural and easy to understand.
