Is it correct to use "they/them" when it's not about people? For example:
I dropped all the books; they lay on the floor.
Yes, it's correct since in your sentence they means the subject of the first sentence- books. And there's nothing wrong in using they/them for referring to non-human objects like books.
But I must say, the sentence seems a bit off. You can alternatively say it as:
I dropped all the books on the floor.
