While this might be seen to be a shortened form of "no ifs, ands, or buts", and certainly this phrase is called to the reader's mind, the words in the passage are a little bit different and more specifically focused.
In context, Nagel is drawing a distinction between our mental constructions of reflection on a purely conditional past event, and of reflection on a free-will choice we have made in the past.
A pure conditional is indicated with an 'if': " You don't mean only that if you had chosen the peach, you would have had it."
But according to him, our idea of a past choice is different! It does not include an 'if'. "I could have had a peach instead" means something else to us. It is not a simple cause/effect relationship where we observe one thing and it implies a second thing.
He's not merely pointing out that the word 'if' does not occur in that sentence; he's saying that what the lack of 'if' points to is that what we mean when we say the sentence about the peach with the 'if' is very different than what we mean when we say the sentence with 'could', with respect to our conception of free will.
This is very different than the usual meaning of "no ifs, ands or buts" - which a speaker would use to indicate conviction of the certain truth of what they are saying, or will not accept questions, doubt, or qualifications.