The Oxford Advanced Learner's Dictionary's article for "all" contains the following example for the section "of all the..."
I've locked myself out. Of all the stupid things to do!
According to the article, the meaning of the expression "of all the..." is "to express anger".
I understand that the second sentence of the two cited above, since beginning with "of all the...", has the purpose to express anger about having locked theirselves out. But how the rest of this sentence ("stupid things to do") is connected to its beginning and to the first sentence? Is that a kind of an elliptical remark?