There are several different kinds of rhyme.
Words that rhyme very closely such as light and sight are perfect rhymes. I think perhaps your statement that certain words do not have a rhyme is referring to perfect rhymes, although a couple on your list are arguably incorrect. There are some lesser-known and/or archaic words such as "chilver" (a female lamb) which rhyme perfect well with "silver".
Words that share a syllable are known as syllabic rhymes. One of your examples is "silver", and this could be rhymed with the word "cleaver" among others. Other terms for different kind of rhymes also exist.
The website Rhymezone is a useful resource for finding all kinds of rhymes. Here are some results from your examples:
silver: chilver, bigger, dinner, figure
purple: argal, argil, barbel, burble, burgle, carpal
month: plinth, lunch, bunch
ninth: smythe, scyth, find, mind