AI-generated art has long been the subject of heated debate. Critics call it soulless and derivative, while enthusiasts hail it as an unprecedented tool for creativity. But what happens when personal biases are stripped away? A recent experiment conducted by the blog Astral Star Codex reveals something: even AI art haters seem to prefer AI creations in blind tests.
The Test
The challenge was simple: participants were shown 50 images and asked to determine whether each was created by a human or an AI. These images spanned Renaissance-style oil paintings, 19th-century landscapes, modern abstract pieces, and contemporary digital art. Some were masterpieces by historical and contemporary human artists, while others were AI-generated by skilled prompt engineers.
With over 11,000 participants, the results were striking. On average, respondents could only correctly identify AI images 60% of the time—barely above chance. Even more intriguing, participants slightly favoured AI art, with six of the ten most-liked images being AI-generated. The top two most popular images? Both courtesy of AI.
Taste vs. Technology
One of the most fascinating aspects of the experiment was that even those who claimed to despise AI art showed a preference for it when unaware of the source. Among the 1,278 participants who rated their disdain for AI art at the absolute lowest on a 1-to-5 Likert scale, the results were paradoxical: even they favoured AI-generated paintings when unaware of their origin. The top two most frequently chosen favourites were both AI creations, and half of their top ten picks also turned out to be AI-made.
"These people aren't necessarily deluded; they might mean that they're frustrated wading through heaps of bad AI art, all drawn in an identical DALL-E house style, and this dataset of hand-curated AI art selected for stylistic diversity doesn't capture what bothers them," shares the Astral Star Codex author, Scott Alexander.
This raises questions about how we judge art: are we swayed by preconceived notions about the medium rather than the art itself?
Some argue that the preference might reflect a bias for specific styles, such as Impressionism, which was well-represented among the AI images. Indeed, AI appears particularly adept at emulating the dreamy, soft-focus qualities of Impressionist art, though the single human-created Impressionist piece included was notably unpopular.
"The human Impressionist painting I included (Entrance To The Village Of Osny, above) was actually quite unpopular. And if we remove all Impressionist paintings, then although humans reclaim the top two spots, an AI is still #3, and the machines still take 40% of the new top ten," shares Scott Alexander in their blog.
The Craft of Convincing AI Art
To ensure fairness, Astral Star Codex curated its selection meticulously. AI images with glaring giveaways—like misshapen hands or nonsensical text—were excluded, as were human works with obvious “tells” of authenticity. This level playing field likely amplified the AI images' appeal, showcasing their best work while minimising their faults.
"I tried to make the test as fair as possible by including only the best works from each category; on the human side, that meant taking prestigious works that had survived the test of time; on the AI side, it meant tossing the many submissions that had garbled text, misshapen hands, or some similar deformity," writes Scott Alexander in their blog. "But this makes it unrepresentative of a world where many AI images will have these errors. I avoided most AI art in the DALL-E "house style", since everyone already knows this is AI - or in other similar styles that humans would have trouble replicating, maybe because they do too much with color and lighting, in a way that few human artists would have the talent or patience for."
However, this isn’t the everyday reality of AI art. The average AI-generated image flooding social media often lacks polish, betraying its origins with odd artifacts and an unnatural sheen. In the test, participants encountered AI creations made by experienced prompt engineers who knew how to finesse the technology to produce truly compelling images. Yes, drawing with AI requires some skill too.
Despite the test results, professional artists and sharp-eyed critics were able to outperform the general public. While the average participant correctly identified 60% of the images, professional artists scored 66%, and those who both hated AI art and were professional artists managed 68%.
These higher scorers often cited “logic” in human art as the giveaway. One artist explained that details in human-created works tend to have an internal coherence—a sense of purpose and placement—that AI images often lack. AI-generated art, while visually appealing, can fall apart upon close inspection, with ornaments, proportions, and textures that seem arbitrary or incoherent.
The Future of Art—or a Glorified Protein Shake?
What this experiment surely highlights is a fundamental tension between AI art’s undeniable technical prowess and the emotional, philosophical, and even economic concerns it raises. For many artists, AI art feels like the culinary equivalent of a perfectly engineered protein shake: convenient, palatable, and even enjoyable—but lacking the soul and depth of a lovingly prepared meal.
Beyond the aesthetics lies a fear that the rise of AI art will flood the market with mediocrity, overshadowing the painstaking work of human artists.
🍿 For those curious to see how they’d fare, the test is still available online.
