The sentence is fine as it is.
The definite article, as you suggest, is only used when something is either unique (which a knife is not) or if it is something previously referred to. So, if the wider text from which your quote comes had already mentioned the knife and who it belongs to, then that is the only context in which you might insert the definite article the way you suggest.
It is more likely to have a possessive pronoun inserted.
The villain... stands sneeringly over our hero, his knife glinting in the sun and ready to deliver the coup de grace.
Omitting a possessive pronoun is very common:
Example:
- He stood there with his knife in his hand.
- He stood there, knife in hand.
Regarding your suggestion of an article, you could say "he stood there with a knife in his hand", because the possessive pronoun that follows makes it very clear that this knife belongs to 'him'. If you were to use the indefinite article in your quote as written, it would be ambiguous where the knife is - it could be randomly lying on the ground.