AI slop fuels concerns over trust in scientific publishing

Facebook
Twitter
LinkedIn
Pinterest
Pocket
WhatsApp

AI slop is increasingly visible across social platforms and, by extension, in places where accuracy matters most. A recent scroll through social media showed how quickly machine-written material circulates, with some users flagging it as synthetic and others sharing it for amusement or to sow confusion.

Early chatbot releases such as GPT-3 popularized the term “hallucination” for confident but incorrect answers. Several iterations later, more capable systems like Claude and Gemini can be useful, but their outputs still require independent verification.

At scale, that poses a challenge for the scientific record, where “trust but verify” becomes a mandate for publishers of peer-reviewed work. That challenge is growing as AI tools are woven into consumer products and personal AI demand continues to rise.

AI slop and the scientific record

In an experiment described by freelance writer and applied mathematician Benjamin Skuse, he attempted to slip AI-generated manuscripts through publisher safeguards to test how resilient those defenses are. According to editors familiar with his effort, Skuse found it surprisingly easy to produce a paper that appeared plausible as science with minimal human intervention.

“He basically hit ‘run experiment’ and ‘write paper.’ And then he didn’t do a damn thing to it.” —Eliza Strickland

Editors said the combination of human review and research-integrity checking tools is keeping pace against work produced with free AI systems. However, they noted that more advanced large language models are avoiding many of the obvious errors common to basic tools.

That has left publishers in an escalating contest between increasingly convincing AI slop and the technologies and workflows meant to detect it. Many of those tools build on work from research institutions and companies such as OpenAI and Google DeepMind, even as the same organizations advance the underlying models.

Wikipedia’s pushback against AI slop

The fight is not confined to journals. Wikipedia, a widely used repository of human-written knowledge, is also trying to stem synthetic content.

In an interview for a recurring Q&A feature, AI editor Eliza Strickland spoke with Ellia Lebleu, a founding member of WikiProject AI Cleanup. The volunteer initiative focuses on finding and removing AI-generated material.

The effort faces the same challenges as commercial publishers, but with fewer staff and fewer tools. “Wikipedia is known as a repository of human-generated information, and they want to hold on to that,” Strickland said.

She added that it remains uncertain how long that stance can be maintained as models improve. The tension between preserving human-authored content and embracing new technology is likely to intensify.

What erosion of trust could mean

Editors and volunteers warn that if the scientific literature or basic factual references cannot be presumed reliable, public understanding risks sliding into uncertainty. That uncertainty may benefit those who exploit information vacuums, but it undermines institutions dedicated to safeguarding knowledge and advancing science and technology for the public good.

Looking ahead, some observers note a possible future in which AI systems both conduct research and draft the papers that other AIs read and evaluate. As Strickland put it, the prospect of AI writing for other AIs raises the unsettling possibility that humans could be edged out of the loop.

Facebook
Twitter
LinkedIn
Pinterest
Pocket
WhatsApp