Roy stopped talking. We sat and stared at the shadows, oversized and faint off the lattice fence.
The author is saying that 'we' can see shadows from the fence.
It is possible that he is trying to show that it is late in the evening, when shadows get very long, but also faint due to lack of light.
It is also possible (not enough context to be sure) that this is set after sunset, and the shadows are from an artificial light - as a 'point source' they can throw shadows that are larger than the item itself, but again, would be likely to be faint if the scene is generally dark.