No it's not wrong.
Life is too short to [do something] is a common expression. You can find examples of usage in the Google Ngram Viewer. Scroll down the page, and use the links to google books to find actual examples of this expression in use.
The expression is not a literal statement, it's an observation about life in general, in the sense that life is too valuable to worry or waste one's time thinking about things which are inconsequential. There is nothing wrong and nothing to fix here. You could add "for us" but it's certainly not a requirement, and to be honest I don't think it adds anything useful here.
Furthermore, it's generally not a good idea to mess around with expressions or try to fix them grammar-wise, as though they were literal statements. Expressions and idioms are generally fixed. Many make no literal sense at all. Of course if you want to be creative there is no rule that says you can't mess around with them for effect. You can, but I would be wary of this. It will rattle an English reader's cage in ways that you may not be aware of. "To rattle one's cage" - look it up!!