It depends on whether or not your regret was still in the process of stemming from these things at the time you put pen to paper—or if these things had already caused as much regret as they were going to.
When I walked through the garden yesterday, I noticed that a flower was blooming.
When I walked through the garden yesterday, I noticed that a flower had bloomed.
Both of those are correct, but they describe different things.
In the first sentence, the flower had started to bloom, but hadn't yet finished doing so. In the second sentence, the flower had already started to bloom and finished blooming. Each of these are possible, and each describe the state of affairs at the time I walked through the garden yesterday.
Which you actually use depends on what was actually the case at that time.
In your sentence, you need to ask yourself if (at the time of putting pen to paper) regret was still being caused by these things (in which case, was stemming from would be more appropriate), or if they had already finished causing you as much regret as they were going to (in which case, stemmed from would be more appropriate).
Also note that in comparing your sentence to my own, you would use had stemmed from, not was stemmed from:
the things I thought this regret was stemming from [Regret was still being caused.]
the things I thought this regret had stemmed from [Regret had stopped being caused.]
Although you could use was stemmed, it wouldn't sound quite as natural.