In my variety of English (Western US English), "I'm little confused/worried/concerned" sound incorrect. We do not typically use little as an adverb in these situations. There are some exceptions, such as "he is little known", but in general, we'd only use "a little" to modify verbs like confused, worried, or concerned.
However, this may not be the case in all varieties of English. Macmillan Dictionary says this is American English:
as an adverb: I go there very little. I’m little affected by the cold
https://www.macmillandictionary.com/us/dictionary/american/little_1
In contrast, your second link, the Cambridge Dictionary, does not suggest that little is ever used as an adverb:
A little: adverb
We use a little as an adverb of degree. It is more formal than a bit:
He smiled just a little.
Her hands were shaking a little.
https://dictionary.cambridge.org/us/grammar/british-grammar/little-a-little-few-a-few
What this means is that, in your context, for speakers like me, there is not a choice between "little confused" and "a little confused"; only the latter is correct.
But if you altered the sentences, so that we were using little as an adjective, we'd end up with a situation that mostly conforms to what you've been taught:
I felt a little confusion -> I felt some amount (neutral) of confusion
I felt little confusion -> I did not feel much (negative) confusion
He expressed a little worry -> He expressed some amount (neutral) of worry
He expressed little worry -> He did not express much (negative) worry