What do I really need to know about colloquial English except phrasal verbs and collocations?
Vocabulary and phrases would also be necessary. Colloquial English extends beyond phrasal verbs in terms of vocabulary. There's a concept called register, which you may already be familiar with, which essentially rank orders words in terms of their formality. They generally fall into three categories: formal, neutral, and informal.
For example, "What's up, mate?" vs "Good evening, Sir" as two types of greeting.
Regarding collocations, I wouldn't say that is really directly relevant to the topic of formal vs colloquial English. It's more about standard/common English. If you don't use typical collocation patterns, then that won't make you sound less formal, but rather more foreign.
"I'm having a party next week" or, "I'm throwing a party next week" not "I'm making/doing a party next week" (A typical 'mistake' Spanish speakers make)