Either “In the bath was a spider,” or “There was a spider in the bath” would be correct. That’s called an existential there. It doesn’t refer to any specific place. It introduces something and says that it exists. However, we can also use was or is with a location existentially.
So, if I know of a good restaurant down the street, I do not think you are aware of it, and I want to inform you that it exists, I might tell you:
- Down the street is a good restaurant.
- There’s a good restaurant. (And point to it.)
- A good restaurant is that one, over there.
- There’s a good restaurant down the street.
You can read 2 as substituting a pronoun for the location in 1, and if I weren’t pointing to a specific location, you could also read it as an existential statement that specified no location. Over there in sentence 3 unambiguously means a location. I could instead say, “down the street.” There’s no substitution for there that works in sentence 4. There in that context is just a dummy pronoun; it does not refer to any place at all.
If you already knew about the restaurant, I would instead say something like, “The restaurant there is good.” I don’t need to tell you that it exists, just specify which one I mean (the one at the location someone had just mentioned).
In modern English, the phrase “there is” doesn’t normally mean the same thing as “it is there.” As you’ve observed, it’s so completely lost any association with place that we can say where something is in the same breath as there is. However, we can substitute a location for there in an existential statement, too.