We can avoid possible ambiguities with a slight rewording:
He was feeling bad, so he went = returned
to work and tried to distract himself.
Here “to” is a preposition, and “work” is a noun. We could replace it with his job and the meaning would remain unchanged.
He was feeling bad; so he went = got down
to work, and tried to distract himself.
Here “to” is the infinitival particle, and “work” is the verb. We could replace it with a different verb e.g., study and the sentence would remain meaningful.
return
- To go or come back, as to an earlier condition or place:
- She returned to her office after lunch.
get down to work is an idiom
To begin being serious about something; to begin attending to business or work at hand