This is hilarious!
At first, I thought your question was about Shame (I can't say the same), but now I can see that it's really about the senses of fit the author used hilariously.
One sense of fit in this excerpt is about the comfortable feeling one feels when being in a place they belong (which is opposite to the feeling of being "out of place", which the author seemed to have when he was away from home).
The other sense is about how well clothes fit someone, i.e. "if clothes fit, they are the right size for you". And fit (someone) like a glove is an expression that means "fit extremely well"
The author was talking about his (I assume that he is male) feeling when he came back home (in the UK). Everything fit him perfectly, even his same old winter gloves he left at home (and of course, this alludes to the expression fit like a glove). However, there is one thing that didn't fit very well. It was his old winter trousers! -- That's an indirect way to say that he gained weight while he was working away from home (for four years).
And that's why (It was a) shame (that) he couldn't say the same of his old winter trousers. :-)