Sometimes it easier to see the correct answer when we use another word in replacement for the one we are having difficulty to understand.
If only we ______________ a three instead of four. We'd be rich!
decide; had decided.
If only we decide a three instead of four. We'd be rich!
This cannot be correct: in this context we will be rich if we decide 3 not 4.
If only we had decided a three instead of four. We'd be rich!
Had is referring to something that was possessed in the past...but
in this context we would have been be rich if we decide 3 not 4.
chose | chosen Ref C.E.D.
to decide what you want from two or more things or possibilities: