I've seen the three versions. The one with a comma, the one with "that," and the one with nothing:
It was so big, it was scary.
It was so big that Sampson had to drag it to her.
Only it was so big it couldn't help making some noise when it moved.
I think this is more a matter of style than grammar (or perhaps it depends on the verb or length?).
Or maybe the comma and the "that" can always be ignored in cases like these?