That day, I had been working in London and I was on my way home when....
or
That day, I had worked in London and I was on my way home when...
(I am on my way home after the working day)
Depends what you want to say
(On) that day, I had been working in London and I was on my way home when....
Has a feeling that you normally don't go to London, but happened to be in London working.
(On) that day, I had worked in London and I was on my way home when...
Has a feeling that you work in London from time to time, and worked in London that day.
(On) that day, I worked in London and I was on my way home when...
Has a feeling that you habitually work in London.