Short version: It must change, but nowhere near enough to become /e/, /æ/ or /u/. The change is not significant enough that you should take it into consideration. In fact, the less you think about it when speaking, the better. Your brain and tongue will take care of the tiny differences for you. :)
I consider this a valid language learning question, because seeing how sounds alternate can be part of a phonological approach to a new language. For example, if an English learner is having difficulty knowing when to aspirate a voiceless stop (i.e. when to say /p/, /t/ or /k/ with the infamous "puff of air"), it will be helpful and probably not too technical to know that this happens at the start of a syllable when it is not in a cluster. That is not to say that this information is necessary for every learner of English. Most people get by fine without it. But it can be helpful.
However, in the case of English schwa, the answer is that it's not very relevant to your learning.
I'll give a little more detail, though, if you're curious (though this part more or less belongs on Linguistics Stack Exchange).
Certainly all articulation is affected by its context. One of the major forces is assimilation. Assimilation is where one sound shares its own features with another sound. There are various reasons, but an obvious one is that we try to save effort. Your tongue doesn't like jumping from place to place too often; your vocal cord vibration doesn't like being switched "on" and "off" too often; your lips don't like to round and un-round too often. So we unconsciously cut corners.
Often, we tolerate the variation. For example, try saying "key" and then "cool", very slowly. If you pay close attention, you'll find that /k/ before /i/ is farther forward, almost palatal, but /k/ before /u/ is farther back, more velar. Your tongue anticipates where it will need to be for the next vowel.
But we are equipped to ignore such small differences. We have to ignore them. The resulting acoustics differ very slightly. In any given conversation, there might be more interference from, say, the pitch of the speaker's voice, whether they have food in their mouth, background noise... So we have an unconscious funnel that takes in a bunch of versions of /k/ and all our brain tells us is that we heard /k/.
In linguistics notation like the IPA, we represent that reality by a kind of fiction, even when being very careful. We write [k] for this whole range of sounds. However, we draw a line for certain features. For example, we write an alveolar [t] differently from a dental [t̪].
Moreover, when we see that in a given language, one "structural" sound is pronounced as different "surface" sounds, we call it allophony. Then, we use the broad notation for the underlying "phoneme": /t/. And we link that to the fine notation for the surface "phones": [t] and [t̪].
The relevant part is this: Most speakers of a language can only ever perceive the phoneme, because their brains have learned that the difference doesn't change the meaning. Learners are likely to be aware of the different phones, because their brains haven't learned to ignore the difference. (For example, English speakers don't notice the aspirated stops in their own language, but they do notice the alternation between [ɾ] and [l] in Japanese even though most Japanese speakers don't.)
And for the things that are too fine to be notated... only acoustic technicians and X-rayers bother with them. :) As far as that goes, I haven't heard of the schwa in English being allophonic. Not in terms of its openness or backness, nor in "lip shape" (rounding?). So don't worry about it.
I will say that in some dialects, there are different sounds we normally call schwa. For example, a US English speaker may pronounce Rosa's differently from Rose's or roses. They might say the first one /ɹoʊzəz/, but the other two /ɹoʊzɨz/ with a reduced vowel similar to the vowel in sit. But this is tied to a change in meaning, so it can't represent the same phoneme.