A man is trying "setting up" (or "to set up") password - difference in meaning?

Question

We know that there's a difference in meaning between try used with the infinitive and try used with the gerund:

I tried sending her flowers. (Meaning: An experiment: I sent her flowers to see what will happen).
I tried to send her flowers. (Meaning: I tried to go through the procedure necessary to send flowers to her. The procedure might have been difficult.)
(per Michael Swan's PEU, Unit 299.6)

Will the same distinction hold if we use trying:

A man is trying setting up a password on his computer. (Is he experimenting, wanting to see what will happen?)
A man is trying to set up a password on his computer. (Is he trying to go through all the necessary steps, probably difficult, of setting up a password?)

Answer 1

The distinction still holds; but (as I suspect you realize) you run into a stylistic problem.

^awk He’s trying setting up a new password.

Two consecutive -ing forms feels very awkward under “The horror aequi principle [...] the widespread (and presumably universal) tendency to avoid the use of formally (near-) identical and (near-) adjacent (non-coordinate) grammatical elements or structures.”^† You would encounter the same problem if you used infinitive to set up with try in the infinitive:

^awk He needs to try to set up a password.

Speakers instinctively ‘rewrite’ to avoid these identical constructions. It actually takes very little to mask the identity. For instance, a stock solution with infinitives is the try and construction—

He needs to try and set up a password

But that won’t work with trying. My guess is that the most likely rewrite with -ing forms is repackaging with a wh- cleft:

What he’s trying is setting up a new password.

---

^† _{Günter Rohdenburg, “Cognitive complexity and horror aequi” in Rohdenburg and Mondorf, Determinants of Grammatical Variation in English, 2003, 236.}

Answer 2

In the present tense "...is trying setting up..." sounds very awkward. Even though it might be correct, I think the meaning would not be fully clear without some analysis on the part of the reader.

In past tense "...tried setting up..." works just fine to my ear.
It implies that he succeeded in that task, whether or not having a password helped him later or not.

Your second example works in present or past tense, implying that he may be having difficulty with the very act of adding a password.