In my opinion either is acceptable, with caveats:
If you have a particular window in mind, then the window refers to that (but the reader wouldn't know which window other than it referring to the one that had been left open).
When you don't know which window yet, then saying a window is indeed possibly more “correct”, but for the reader, nothing has changed: we still don't know which window (unless you were writing a book and were able to add illustrations). There is also the problem that it can parse a bit more like realized I had failed to close a [single] window ... meaning I had left all the windows open.
I'd probably say had forgotten to close (the|a) window myself, but it's in no way “better” than your version(s).
The only bad option would be to say a window if you only had one window in the house :o).