This is venerable rather magnificent institution
is wrong for grammatical reasons. It is also ambiguous even if we fix the errors in grammar because it could be made grammatical in two completely different ways. For example,
This institution is venerable rather than magnificent
is grammatical and equates to
This institution is venerable but not magnificent
But the following revision is also grammatical
This institution is venerable and rather magnificent
which equates to
This institution is venerable and somewhat magnificent.
Somewhat and not are quite different meanings.
The problem is that rather than is a conjunction that means but not.
But rather used as an adjective means to a partial degree.