In terms of a standalone sentence, the singular makes sense:
✔ The other day, I met a clown on the street.
But as a standalone sentence, the plural is unidiomatic:
✘ The other days, I met a clown on the street.
However, in the right context, the other days is perfectly acceptable:
✔ On Monday last week, my walk to work was uneventful. The other days, I met a clown on the street.
There is nothing unusual about that. It's a perfectly normal application of words, with a perfectly understandable plural usage. (And it's understood that the other days means the other days of the week.)
In the acceptable version, the other days is used in contrast to the one day (or it could be in contrast to multiple stated days) when something different happened.
Further, it's actually the use of other that requires some kind of contrasting statement, not days itself:
✘ The other pieces of fruit were fresh.
✔ One piece of fruit was rotten. The other pieces of fruit were fresh.
So, your sentence is perfectly fine—as long as it's preceded by something else that indicates one or more days when it was not raining:
✔ That week, it was a nice day on Monday. Unfortunately, it rained the other days.
If the first sentence is left out, the second sentence is still grammatical, but it becomes confusing.