I understand why you think it is vague and strange — this sort of writing is quite evocative rather than explicitly descriptive.
Jigsaw here, in describing geometry, is used to give a sense of “many things that fit together”. So “crazy, jigsaw geometry” is perhaps an unbelievable, haphazard arrangement of many things which fit together to give some overall sense of shape or structure.
Then bringing in the decay, this basically tells you what those pieces of the ‘jigsaw’ are. In this case, it means “things that are in need of repair or attention” or “things that are run down”. Everywhere there seems to be many small instances of (urban or natural) dilapidation, all fitting together. The author then gives examples of the decay: the broken pavements, the concrete slabs, and the yellowing neem leaves in the sand.
The way this passage is written gives a sense of awe and beauty, that all of these smaller instances of decay fit together. In themselves may not be beautiful or impressive, but collectively, somehow, it is.
Does that help? I hope so!