The first sentence ("He as well as I were invited to the party") just doesn't sound right to my ears.
As far as the technicality of grammar is concerned, I think whether the subject is noun or pronoun plays no role in the determination of whether to use singular or plural verb. Rather, what matters is the number of the subject.
While “as well as” can mean “and”, it in the context of the given sentences means “in addition to” (as opposed to “in addition” or “and”).
And, where “as well as” means “in addition to”, what comes after “as well as” doesn’t change the number of the subject. Therefore, the number of the subject is the number of the item that is before the “as well as”. In other words, irrespective of the presence of “as well as + whatever”, the verb must follow or agree with the number of what’s before the “as well as”. Why? Because here “as well as” means “in addition to” which is NOT a conjunction.
So, the technicality of grammar knocks out the first sentence ("He as well as I WERE invited to the party") because of the use of the plural verb “WERE” since “He” the controlling part of the subject is singular.
Irrespective of the technicalities of grammar, we still can accept something as grammatical when lots of native speakers popularly and practically use that. But I can’t remember hearing too many people using “WERE” in the identical types of situations.
Accordingly, whether talking about the natural use in terms of the way I am used to or the technicality of grammar, what seems grammatical to me is ONLY:
“He, as well as me/I, was invited to the party.”
NOTE
For written English, we should insert commas as reflected above.