Substack's latest attempt to help readers tell human writing from AI-generated text has instead sparked accusations that the platform is turning writers into suspects.
The newsletter platform has begun rolling out an AI detection feature powered by Pangram, allowing readers to estimate how much of a post appears to be written by artificial intelligence versus a person.
While Substack says the feature is meant to improve transparency as AI-written content becomes more common online, many writers argue that it risks doing the opposite by encouraging readers to question authors before they question the technology.
Critics have described the feature as a witch hunt, citing persistent accuracy issues that have plagued AI tools.
Rather than settling the debate over AI-generated content, the rollout has exposed a broader question online: whether automated detection systems are reliable enough to be used at scale in settings where errors can carry real human consequences.
The trust tool that was opposed by its own beneficiaries
The backlash has come from an unexpected direction: the very group Substack says stands to benefit the most from this.
According to 404 Media, many writers have called the feature a witch hunt, arguing that it could change how readers approach every article. Instead of focusing first on quality, critics fear readers may begin by questioning whether a human wrote the piece.
“I’m not going to apologize for using AI in the creation process,” a contributor identified as Mack Collier said. “I wrote for 20 years without AI, I could do it again if I wanted to.”
According to Collier, AI helps structure writing and increase output, while also serving as an editor.
Writers who don’t use AI are also concerned about the tool falsely labeling their work. Alice Lemee, a ghostwriter and digital writing coach, said one false accusation could leave a writer’s reputation “almost irreversibly tarnished.”
The cost of AI getting it wrong
The backlash also revives a familiar criticism of AI: confidence does not always translate into correctness. While Pangram is designed to detect AI-generated writing, it is, at its core, an AI system that makes probabilistic judgments about how a piece of content was created.
Pangram CEO Max Spero has defended the tool's performance, saying the company estimates its false-positive rate at roughly one in every 10,000 scans.
On paper, that appears to be an exceptionally low error rate. But that figure looks different when applied to a platform the size of Substack, where millions of posts, comments, and notes could eventually be scanned. Even a tiny percentage of mistakes could translate into a meaningful number of writers being incorrectly flagged.
Google’s AI search products have faced similar scrutiny after an experiment showed that Gemini and AI Overviews could repeat planted false information, illustrating how errors can become more consequential at scale.
The broader lesson goes beyond Substack
The dispute over Substack's AI detector ultimately reflects a wider question facing technology companies and enterprises alike: how much trust should be placed in automated systems when their decisions can affect real people?
The same concern has surfaced repeatedly as AI takes on larger roles in production systems.
Companies across industries are expanding AI into customer support, fraud detection, recruitment, and other workflows where a single incorrect decision can carry reputational or financial consequences.
Discord, for example, recently faced criticism after an automated moderation system mistakenly penalized legitimate users.
Substack's rollout adds another example to that growing list. Whether Pangram's detector proves highly accurate or not, the backlash shows that users are often less concerned with how frequently an AI system gets things right than with what happens when it gets something wrong.
Right now, the challenge is no longer simply deploying AI. It is about deciding how much autonomy those systems should have in production environments.
Also read: GPTZero flagged suspected AI content and unreliable citations in four PwC Middle East reports, and the firm said it was updating a limited number of supporting citations.