Yes, this is perfectly correct. It's not uncommon to say things like this when speaking naturally, and you find yourself using a verb with a preposition (like "stay in") along with an adverb of time or place (like "in the evening"). If you're quick (and you care about such things) you could replace "in the evening" with "this evening" and avoid the double "in", but generally no one will care.
Other examples:
Please stand by by the phone.
Would you like to hang out out of the house?
In the evening traffic tends to ease off off of the main roads.
And many others.