Cherry picking in artificial intelligence research (and in some other contexts) is the act of choosing examples of good (accurate, grammatical, etc.) predictions. I wonder which word or phrase expresses the contrary of "cherry picking" is (i.e., choosing examples of bad predictions, for example to see what kind of mistakes the model makes).
asked Jun 27, 2023 at 22:56
-
Can you clarify what you mean by "good" and "bad"? In my experience, "cherry-picking" primarily means intentionally selecting examples that support one's own argument while ignoring or downplaying counterexamples. The opposite, therefore, might be intentionally selecting examples that undermine one's own argument. I'm not sure why anyone would do that (perhaps in order to play devil's advocate?), but is that what you mean?– MarcInManhattanCommented Jun 28, 2023 at 0:19
-
@MarcInManhattan e.g., for a question-answering system, a good prediction is a good answer, and a bad prediction is a bad answer. Selecting bad predictions is useful to see what kind of mistakes the model makes.– Franck DernoncourtCommented Jun 28, 2023 at 0:31
-
OK, and by "good/bad prediction" I think you mean "accurate/inaccurate prediction" (as opposed to a prediction of a beneficial/harmful outcome for somebody, for example). I'll try to write up an answer.– MarcInManhattanCommented Jun 28, 2023 at 0:35
-
@MarcInManhattan thanks, in that case inaccuracies are indeed the most common issue, but an answer could be bad for another reason such as repetitions, being overly verbose or a grammatical issue.– Franck DernoncourtCommented Jun 28, 2023 at 0:43
-
1I added some of your comments into the question to try to make it a bit clearer. Please feel free to change or remove those edits if I got anything wrong.– MarcInManhattanCommented Jun 28, 2023 at 1:02
|
Show 2 more comments
2 Answers
I do not believe that there is a specific term for that in general English. (Specialists in certain fields might have specific terms for it, though.) M-W defines the verb "cherry-pick" as:
to select the best or most desirable
In this case, "best or most desirable" refers to what is most desired by the cherry-picker. If the person wants to select only "bad" predictions, then those are the "best or most desirable" predictions for that individual and that individual is still cherry-picking.
By the way, M-W's thesaurus offers some antonyms for "cherry-pick", but they consist of words such as "refuse", "reject", and "decline". None of those has the meaning of selecting anything bad.
answered Jun 28, 2023 at 0:58
If you do "proper" random sampling, then you don't cherry-pick. Indeed "sampling" on its own suggests a statistically sound method of selection has been used.
So "I took a random sample of predictions" would imply that you were not cherry-picking.
