The only one of these that sounds right to me is "I have a guilty conscience." That is a very common expression. In real life, though, I would usually just say "I feel guilty" or "I feel bad about " whatever.
I've never heard the phrase "guilt complex" before.
"Guilt trip" refers to when someone else makes you feel guilty, often undeservedly, but not always. For example:
"My mom is sending me on a guilt trip because I can't come home for Christmas."
"Guilt trip" can also be used informally as a verb:
"My mom is guilt tripping me into coming home."
Meaning my Mom is convincing me to come home by making me feel guilty.
Hope that helps!
