In the dictionary, 'atemporal' is often paraphrased as 'tenseless'. It thus seems that 'atemporal' is identical in meaning with 'tenseless'. Can we then speak atemporal language instead of tenseless language?
Time is the thing that we refer to. One way of showing that something happened in the past is to change the form of the verb. So in English we can put an -ed ending on the verb, if we want to show we are talking about past time.
When we change verbs like this, the verb has tense. This means it has a special grammatical form, and that we usually use this form to indicate which time we are talking about.
So ageing—growing old—for example, is a temporal process. It is related to time. But it isn't related to tense! Ageing is about time and biology, not grammar!
So being atemporal means not related to time or not changed by time. Tenseless means not having a grammatical shape used to indicate time or modality.
An infinitive is tenseless. It is not marked to show time reference.
"Two squared equals four" describes an atemporal relationship. (It is not related to time in any way.)
As with any adjective, the exact meaning of the adjective atemporal is defined by its use in context. There are three main ways (that I know of) that language is often described as atemporal:
A particular text may not refer to any particular time. So, for example, some stretches of Latin text used by the Christian church avoid referring to past or future time (by using participles and infinitives only. In this case , there is also no tense). So these stretches of text can be described as atemporal.
A specific language (as opposed to some words used in a particular situation) may be described in an atemporal way, in the sense that the person describing the language does not recognise that the language changes over time.
An artificial language may have no means of referencing time, and may therefore be considered atemporal.