This is a cliche remark of a stand-up comedian facing an unresponsive audience. The idea is that from his point of view on stage an audience sitting motionless and silent in the dark are hard to distinguish from an oil painting.
What is this, an audience or an oil painting? -- Milton Berle
Bruce Forsyth (concerned): You don't want to lose your audience.
Fozzie Bear: You — you call that an audience? It looks more like an
oil painting. (rimshot) Aah!
The Muppet Show
Comedians have what is known as “save” lines. These are either
self-deprecating or audience-insulting lines that acknowledge that
things aren’t going so well. Lines like: “Is this an audience or an
oil painting?” “Did I mention I have videotape of ALL these jokes working?”
“Was there an ether leak in here?” “I’ve got a brand new handgun, and
I can’t wait to get home and use it.” (These lines are a combination
of my own, one is from Lord Carrett, and the first is older than
How To Deal With Rough Crowds
I don't think it's relevant that the expression is used with (or without) the word "no," as another answer suggests; or that it relates to whether the people in the audience are good-looking.