Yes, "upstairs" is often used to describe a floor above the one you have previously mentioned, even in buildings with many floors.
I can't find fault with your example, yet it doesn't feel quite right because you state right at the beginning that the people are on the third floor. As you must have already stated that the gunfire is occurring on the 10th floor, to then add "upstairs" at the end of the sentence is something of a redundancy.
As an alternative, you could say:
The office workers on the third floor could hear the gunfire from upstairs.
Office workers said the gunfire could be heard from as far away as the third floor.