Grammatically speaking, should there be a comma before "of course" when it's preceded by a conjunction?
"First, let me say that, while punctuation and grammar are related, details like commas don't generally make something ungrammatical." - JavaLatte (Grammar, punctuation, comma)
The commas may change how the sentences are read, but they do not affect the grammaticality of your sentences.
(Maybe there should be a comma because "of course" is a parenthetical phrase?)
Whether something is parenthetical (i.e., nonessential information) or not depends on the author (i.e., what the author wants to convey).
He ordered a batch of cup noodles and other instant food. And(,) of course, beer.
Note that you either put both commas (to set off "of course" as a parenthetical element in the middle of a sentence) or you don't put any at all if you don't intend to set it off as a parenthetical element.
The single comma does not make much sense (to me); it is not required because the sentence is quite small.
There are three reasons to put a single comma: (1) To indicate a pause in speech, (2) to emphasize or draw attention to "beer", and (3) to indicate that "And of course" is an introductory phrase (as pointed out in comments by userr2684291).
I personally would not use a single comma to indicate that "And of course" is an introductory phrase, given how small (one word - beer) the rest of the sentence is. If I were to use a single comma, it would be either for (1) or (2).
Yes, if you want to indicate a strong break in thought, use the pair of commas to set off "of course" as a parenthetical element in the middle of the sentence. If you find yourself emphasizing "of course" when you are reading out the sentence (i.e., if you are changing the tone a bit), then you are actually looking to create some sort of an effect with "of course". If this is true, use the pair of commas. This will indicate the break in your thought or the change in your tone.
If your tone is relatively unchanged and smooth, and if none of the three reasons above for a single comma applies to you, then "of course" is just filler material, and you don't need commas. If at least one of them applies, then use a single comma.