Many core English words words (the short ones derived from Norse or German) have multiple meanings... According to the Cambridge Dictionary, the word take has fifteen meanings as a verb, one of which is
to move something or someone from one place to another
Similarly, off has multiple meanings, one of which is
away from
So it's not unreasonable for take off to mean moving away from the ground. Note that take off does not mean "flying, or rise", According to the Cambridge Dctionary, it means
If an aircraft, bird, or insect takes off, it leaves the ground and begins to fly
So there is a little logic to phrasal verbs... but not much. Phrasal verbs are considered distinct because they have a specific meaning beyond the general meaning of their component parts. To quote the Cambridge Dictionary definition:
a phrase that consists of a verb with a preposition or adverb or both, the meaning of which is different from the meaning of its separate parts
So yes, you do need to memorize them as distinct items. Please refer to a good dictionary to get an accurate definition, and be aware that many phrasal verbs have multiple meanings.