First, let's look at a similar sentence:
1a. [ All I wanted ] was 〔an ice cream cone〕.
1b. [ The only thing I wanted ] was 〔an ice cream cone〕.
In example 1a, all I wanted is a noun phrase.
It means the only thing I wanted, so example 1b means the same thing.
Your example is similar:
2. [ All I did ] was 〔hand someone a bag〕.
Here, the bare infinitival clause hand someone a bag is used as a complement of specifying be.
This is one of the few functions of bare infinitival clauses—they appear mainly as complements of a few specific verbs, and one of those verbs is be.
If you'd like, you can call it a nominal bare infinitive clause, because it appears where a noun phrase would normally be expected. But in any case, this construction is grammatical and totally normal.