- We could not imagine how the Inca could have done that.
(let's just concentrate on the modals here, OK?)
is grammatical, and means the same as the sentence
- We could not imagine how the Inca could do that.
Both refer to past times, but the first one overtly specifies the time of the Inca's actions as prior to the time of our failure to imagine them, while the second one doesn't.
In a case where the context makes the order of events obvious (the Inca's actions are in the far past, whereas we're still alive), it's not necessary to specify it overtly.
If you do specify that order when you don't need to, you draw attention to it. This may be what you want to do; or it may be unnecessary detail that slows down the reader/listener's understanding of your point.
The choice is, as usual, up to the writer/speaker, who has a better idea than anybody else of what they want the reader/listener to understand, and how to achieve it.
Neither sentence is subjunctive, by the way. Just more modal auxiliary usages, that's all.