The Oxford dictionary says that it means plentiful. Can it be used to express a connotation like "Very talented and prolific singer"?
“Very talented and prolific singer” is a perfectly normal use of the word prolific. It means a singer who wrote or performed a lot of songs (prolific) and who did that well (talented).
Prolific alone is only an indication of quantity, not quality, even though prolific tends to be slightly laudative. You can say that someone is a “prolific but mediocre singer”.
Note that prolific generally means “which produces a large amount” or “which gives a large amount”, not “which contains a large amount”.