I have always (since my earliest memories) considered the official name of the nation to be:
- The United States of America
For instance: I am a citizen of The United States of America. (Notice that I capitalized "The", even though it appeared mid-sentence, because it is part of the name.)
However, I also recognized the following as also being valid shortcuts:
- United States of America
- United States
- America (although I later learned that this is somewhat unclear: referring to "the Americas" is a way to specify both North America and South America, and "American" might refer to something related to "the Americas", particularly if the discussion is making contrasts to European or Eastern cultures)
- USA (abbreviation for United States of America)
- The United States
- U.S. ("you ess")
- U.S. of A
Based on the above, saying "the USA" (starting out lowercase) is appropriate, because it is a shortcut referring to the informal name of "United States of America", while "The USA" (starting out uppercase) is also appropriate, because it is a shortcut referring to the full authentic name. I don't typically consider either of these abbreviations to be wrong.
This nation also has multiple names that are valid names (even if they are not the one single official name), just like I have a full name (which includes a "first name", and a "middle name", and a "last name"), but I also have a shorter name that I typically go by. (I'm referring to my first name, which my parents have frequently called me. I'm not even referring other pseudonyms/names/nicknames, like "TOOGAM", that I may have used over the years.)
I wouldn't typically say, "he traveled to United States of America" (entirely leaving off the word "the"), just like I probably wouldn't be very likely to say "he traveled to Hawaiian islands" or "he traveled to Twin Cities". Maybe I could say some of those things (and not completely sound grammatically incorrect), but I would be more inclined to say "he traveled to the Hawaiian islands" or "he traveled to the Twin Cities". In these cases, the names are Hawaii and "Twin Cities", and so I leave the word "the" in lowercase because it isn't part of the actual name. Similarly, if "the" is not being treated as part of the shorter "United States of America" name, then it should be lowercase because it is not part of the name that is being used. However, if "The" is being treated as part of the longer full name, which is "The United States of America", then it should be uppercase because it is part of the name.
Both names are widely supported, and so I would say that you should be able to get away with either choice (regarding whether to capitalize the letter). One way, "The" is part of the title. One way, "the" is not part of the title. Both ways sound the same when being pronounced. When speaking, it can be impossible to tell which way the person had in mind, but that's okay because it really doesn't make any difference to what the person was trying to say. (Also, the speaker probably wasn't bothering to even think of such a distinction.) If two people are writing down the words spoken by one person, and each of those two people capitalized that word differently, I would consider each of the resulting documents to be an accurate record of what was spoken, word-for-word.
In truth, the term "The United States of America" is not used in common speech nearly as frequently as the shorter names. We typically leave off the "of America", or leave off "The United States of". We will sometimes use "US" or "U.S." as an abbreviation. (For instance: "He went to Europe, and then got back to the U.S.") When actually writing things down, I've found that to be getting to be less common than "USA". I guess the single extra letter is considered worthwhile, just to be less likely to conflict with a possible two-letter abbreviation for something else. So, when writing, we will often use "USA" because it is shorter than "America" or "United States". We use the abbreviation based on the full name, although when we don't abbreviate then we tend to use a shorter name, unless we really want the entire name (probably because we are trying to be as formal as possible).