John says he would rather be carrying a pen.
means he is carrying something other than a pen, and he would prefer that he were carrying a pen. (I'm replacing your first He with John to make it easier to refer to him in the text below.)
The time that is being talked about is ambiguous. Sorry, but this is going to be complicated.
The word says is in the present tense, but in English, we seldom use the simple present to mean an action happening right now. Here, says means that the sentence describes John's current preference, not that right now he is in the act of stating his preference. John may have said this yesterday, or last month (or even centuries ago, depending on context). The sentence means: "Some time in the past, John said his preference, and he prefers to be carrying a pen."
The word be is infinitive, so it doesn't have a tense. However, it's combined with carrying, so it forms be carrying, a continuous infinitive. This suggests that John is carrying something other than a pen right now, and would like to carry a pen instead right now, but it's still not completely unambiguous.
The word would is a strange word in English: technically, it's the past tense of will, but you use would for past, present, and future. This is because in English, one use of the past tense is to describe a hypothetical or imaginary situation in the present or the future. The sentence describes an imaginary situation that John prefers, so would doesn't indicate the time when that situation is imagined to exist.
So, without context, the continuous infinitive be carrying suggests that John's preference is that he would like to be carrying a pen right now, but the time is not clear.
With context, though, it can become clear. Here are some contexts where your sentence would make sense.
Here is an old movie of John walking along the street, carrying a shovel. In the movie, John says he would rather be carrying a pen. [The imagined pen-carrying happens in the past for the person speaking, but in the present for John in the movie.]
Here is the latest version of our movie. As you can see, John is carrying a shovel. I talked to him this morning, though, and he says he would rather be carrying a pen. [The imagined pen-carrying occurs in an imaginary modification of the current version of the movie.]
John has been carrying a shovel to work every day since 1999. He says he would rather be carrying a pen. [Here, the time is the habitual present. John imagines that his current job were different, so that his habitual behavior right now would include carrying a pen instead of a shovel.]
John says he would rather be carrying a pen in 2029. [Here, the imagined pen-carrying is 14 years in the future. The continuous aspect suggests that the imagined pen-carrying is habitual behavior in 2029.]
The lesson to learn from all this is that the infinitive doesn't have a tense of its own, and the time it refers to adjusts to suit the context.
This sentence is ungrammatical:
John says he would rather carrying a pen.
It's ungrammatical because would rather takes an infinitive or a subjunctive clause, not a gerund. You can say "John says he would rather be carrying a pen" (infinitive) or "John says he would rather that he were carrying a pen" (subjunctive) but not the version with a gerund. I don't think there is any useful general rule to know about why this is. The important thing to know is: in English, each standard phrase has its own customs for which other phrases it will combine with.
These sentences are grammatical:
John says he would prefer carrying a pen.
John says he would prefer to be carrying a pen.
John says he would prefer that he were carrying a pen.
But this isn't grammatical:
John says he would prefer be carrying a pen.
I recommend that you not memorize the grammatical terms and rules. Instead, memorize the phrases. This is how English speakers do it: they make new phrases by analogy with familiar phrases, not by following rules.