Both "went over" and "went over it" are right, but they mean entirely different things, and it's likely you want to use a third alternative altogether.
- She stood up, and went over. — She stood up and walked to the bookshelf.
- She stood up, and went over it. — She stood up, and looked carefully through what was on the bookshelf. Here, "go over" means "to analyze, scrutinize, have a thorough look". It is not said that she had to make a single step towards the bookshelf.
- She stood up, and went over to it. — Same as in (1), but you are being more specific about the fact that it was indeed the bookshelf she walked over to, and not something else.
As you can see, all three make sense as such. She either scanned the room, then proceeded to scan the bookshelf; or she scanned the room, then went to the bookshelf and nothing is said about why she did that or what followed (the next sentence would likely clear that up). If I wanted to express the latter, i.e. choose between (1) and (3), I'd go with (3) for reasons of clarity and style.
Edit: Now that you've provided the next sentence — "then, as if she were searching for a secret passage, she examined each books" — it is rather clear to me that (2) is not really an option here, as you'd be saying "she examines the room, then the books on the shelf, then the books on the shelf again". So you really are looking to use (3) here. She examines the room, notices the bookshelf, walks over to it, and examines each book. (Do note that it should be "each book", not "each books".)