It's nothing to do with sitting. Prepositions are often governed by a particular verb or adjective, but in this case, it's their primary locative sense.
At tends to imply more than just physical location: either that the person is there for a particular purpose ("sitting at the table"; "waiting at the checkout") or that the place is distinctive, and saying it defines where the person is ("at the corner"; at the London Eye").
So if you said they were sitting at the glass wall, it would either suggest that they were doing something with the wall (may be they'd sat there so that they could look through it), or that the glass wall was a landmark you expected your hearers to know.
By doesn't have these connotations, so that would be the normal choice here.