My heart grieves with the thought that he will stay back.
My heart grieves at the thought that he will stay back.
Grammarly corrects the first sentence to the second one. However, I have come across the first structure a lot more than the second. At the thought and With the thought—can these two be used interchangeably? Or do they mean different?