AI may not have a soul, but readers are increasingly willing to give its stories a five-star review.
A new study found that participants rated three short stories generated by ChatGPT 4.0 as more absorbing and higher quality than three comparable human-written stories, challenging the assumption that human authors always produce more compelling creative work.
The research, published in Judgment and Decision Making, involved 1,682 adults who read one of six short stories. Three stories were written by human authors, while three were generated by ChatGPT with similar themes to the human-written works.
Participants were told whether each story was written by a person or AI, but researchers intentionally provided incorrect labels in some cases. They then rated the stories based on quality and how absorbing they found them.
All six were roughly 1,000-word works of realistic fiction, and the participants were U.S. adults recruited through the online research platform Prolific.
The results showed that AI-generated stories received higher average ratings than human-written stories. However, the label attached to the story also influenced opinions: readers gave higher scores to stories they believed were written by humans, even when those stories were actually generated by AI.
Dr. Deena Skolnick Weisberg, senior author of the study from Villanova University, said the findings show AI’s creative abilities may be underestimated.
“AI systems can already generate short stories that are seen as being at least as good as – if not better than – human-written stories. We should update our views of AI’s abilities accordingly,” Weisberg said, according to The Guardian.
Most readers could not identify AI-written fiction
The researchers also tested whether people could reliably distinguish between human and AI writing.
In two additional experiments involving 905 participants, readers were given two stories with the same theme — one written by a human and one generated by AI — and asked to identify which was which.
Participants performed poorly. In one experiment, only about 40% correctly identified the authorship, while another produced a 52% success rate, suggesting people were generally unable to distinguish AI-generated stories from human ones.
The study found that people with more experience using AI tools were better at spotting AI-generated writing. However, people who considered themselves more knowledgeable about fiction did not perform better at identifying human-written stories.
The real challenge may be trust, not quality
The findings point to a growing gap between AI’s technical abilities and public perceptions of those abilities. While AI-generated content can compete with human writing in terms of readability and engagement, many readers still place extra value on knowing that a person created the work.
Researchers found that attitudes toward AI played a role in how people judged stories. Participants with more positive views of AI were more likely to rate stories they believed were AI-generated more favorably than those with less positive views.
Weisberg argued that stronger AI writing does not eliminate the value of human creativity.
“We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn’t mean that there’s no longer space for us to appreciate the process of human creativity,” she said, according to The Guardian.
What this means for writers and businesses
The study suggests generative AI could become a more common tool for storytelling, marketing, entertainment and other creative industries. Companies may increasingly use AI to produce drafts, experiment with ideas or create content at scale.
But the research also highlights a major limitation: audiences may care about the origin of content as much as the content itself. For publishers, brands and creators, transparency around AI use could become an important part of maintaining trust.
AI-generated writing may be improving quickly, but the human connection behind a story remains a powerful part of how people value creativity. The future of writing may not be a competition between humans and machines, but a mix of both — where AI handles some of the production while humans provide purpose, experience and perspective.
Also read: OpenAI is offering free ChatGPT access to as many as 100,000 academic researchers, expanding how scientists can use its advanced AI tools for research.