"Should I use the past tense with did"
Answer: For the sentences you posted:
Did they knew what it means or not?
Did they know what it means or not?
The answer is this: any interrogative in English (except for the verb to be) is formed like this: Helping verb + subject + main verb.
The simple present tense, declarative: They know what this means
The simple present tense, interrogative: Do they know [etc.]
The simple past tense, declarative: They knew what this means
The simple past tense, interrogative: Did they know [etc.]
Therefore, /Did they knew/ is not grammatical.
For irregular verbs (except for be), the simple past interrogative form uses DID + the subject + MAIN VERB.
DO and DID are called the helping or auxiliary verbs used in Interrogatives.
The other question: /All I did was x/ is very different from what I explained above.
The DID in /all I did was/ is the past of the the main verb to do and is not a helping or auxiliary verb even though it is written the same way, the function of it is not the same:
Examples: All I do is teach English (present tense). Declarative.
All I did was teach English. (past tense of to do). Declarative