As BillJ says in a comment, difficult resists taking a passive infinitive complement.
But I think this is primarily a semantic restriction. When you use difficult to X, the presumption is that the subject of X is the entity who experiences difficulty:
I find it difficult to walk slowly.
So if you used a passive:
Paper-based books are difficult to be stored
it means that the books are finding it difficult, not that you are.
With suitable verbs and context, you can use a passive infinitive:
It is difficult to be heard, with all this noise around.
I don't think this is very common - most people would say something different, such as It is difficult to make yourself heard; but it is possible.
You can, of course, also say
It is difficult to hear, with all this noise around.
but that is from the perspective of the hearer, rather than the speaker.