I met both variants and I feel that there is an important difference between them? But I cannot pinpoint it
1 : something conceived in the mind : thought, notion
2 : an abstract or generic idea generalized from particular instances
"Concept of the universe" uses the first definition and "concept of universe" uses the second.
Here are a couple of example usages:
In my astrophysics class, we studied the concept of universe.
In other words, we studied the abstract idea "universe".
When I realized how small I was, I had a better concept of the universe.
In other words, I had a better thought or notion of what the universe was.