
The Rise of Citation Slop
A groundbreaking study finds that AI-fabricated references are a growing problem in biomedical research.
In late 2025, an artificial intelligence (AI) tool provided Maxim Topaz, PhD, with some help he would rather not have gotten. Topaz, Columbia Nursing’s Elizabeth Standish Gill Associate Professor of Nursing, is a leading expert on the use of AI in patient care. He and his colleagues had just completed a research study and were preparing it for publication in a major journal. The manuscript had been written entirely by its human authors, who’d proofread it repeatedly prior to its submission.
The problem emerged after the journal asked that the manuscript and several other files be combined into a single PDF. Topaz and his team used AI for the task, an increasingly common approach in recent years. Soon afterward, he received an alert from an editor that one of the paper’s citations didn’t lead to a source. It turned out the citation had been introduced not by the authors but by the AI tool. “The AI must have thought, ‘Hey, this is a good thing to insert,’” Topaz explains. “It was trying to improve on our work.”
The irony, he adds, is that the team had deliberately kept AI out of the citation process, aware that such tools can introduce plausible-sounding falsehoods—commonly known as hallucinations—into research papers. “Yet a fabricated reference still snuck in, because of a routine formatting step that few researchers would consider risky. And it was caught only because that journal happens to verify references, which most do not.”
Topaz was both embarrassed and alarmed by the incident. “I thought, ‘If this happened to me, it could happen to anyone,’” he recalls. It also made him wonder: How widespread is this kind of fakery? Although other observers had noted the phenomenon, often dubbed “citation slop,” no one, he soon found, had studied it.
He set out to find the answer. The result, published in The Lancet in May 2026, was the first systematic study of the authenticity of references in scientific literature—and a wakeup call about the potential impact of imaginative machines on the reliability of healthcare data.
To investigate the problem, Topaz assembled a multidisciplinary team with four other researchers across the Columbia campus and in Israel and Finland. He and his coauthors developed an automated verification system capable of distinguishing citations containing innocent errors (such as inaccurate titles or misattributed authorship) from those invented from scratch. They then
scanned 2.5 million biomedical research papers, published from January 2023 to February 2026 and listed in PubMed Central’s Open Access archive; altogether, the papers contained 97.1 million references, which they analyzed using their custom-built tool.
“We flagged suspicious references first, then put them through sequential filters to minimize false positives,” explains one of Topaz’s postdocs, Zhihong Zhang, PhD, a nursing informatics researcher who helped test and evaluate the system. “Those that passed all the filters were checked against PubMed, Crossref, OpenAlex, commenand Google Scholar. If they didn’t show up in any of those databases, we labeled them as fabricated.”
Among the filtered references, the researchers identified 4,046 fake citations in 2,810 papers. The incidence rose sharply after mid-2024, coinciding with the widespread adoption of so-called “large language models,” or LLMs—the type of AI that, unlike older models, can help humans write or edit documents, rather than simply discern patterns in masses of complex data. In 2023, 1 in 2,800 papers contained at least one fabricated reference; by early 2026, the ratio was 1 in 277. The rate of fabrications rose more than 12-fold, from 4 per 10,000 papers in 2023 to 51 per 10,000 in the last quarter of 2025. It had reached 57 per 10,000 by early 2026.
The fakes were often uncannily convincing: perfectly formatted, plausibly dated, and credited to authors known for related work. One paper the team reviewed had 30 references, 18 of which were phony. Some of those fabrications, they found, were already being cited by other papers and appearing in systematic reviews. And in over 98% of cases, the papers with fake references had received no correction, retraction, or expression of concern.
“Because clinical guidelines are based on evaluations of published research, fabricated references can directly impact patient care,” Topaz notes. “Nurses and doctors may increasingly be making treatment decisions based on studies that never happened.”
To combat the rise of citation slop, Topaz and his coauthors recommended four actions. First, they advised publishers to integrate automated reference verification into submission workflows even before the peer-review process begins. Such tools, they pointed out, already exist, though few publications use them. Second, they suggested that indexing services (such as Scopus, MEDLINE, and Web of Science) add integrity metadata to article records so that downstream users can assess the reliability of references more easily. Third, they encouraged publishers to retroactively screen existing publications and issue corrections or retractions when fake references that compromise a paper’s conclusions are uncovered. Fourth, they urged research integrity databases to establish “fabricated references” as a dedicated category, enabling systematic tracking and accountability.
“A citation is a promise that evidence exists,” Topaz says. “When the evidence is fake, that promise is empty. As things stand, however, policymakers and healthcare providers have no way to know.”
Topaz didn't start out as a critic of artificial intelligence in healthcare. In fact, he’s spent more than a decade researching ways to leverage it. But his commitment to optimizing AI’s potential and his concern that it be developed and deployed responsibly grow from the same root: a passion for improving patient outcomes.
He was drawn to that cause as a boy in post-Soviet Russia, when his 83-year-old grandmother was turned away from a clinic after breaking her hip because doctors considered her too elderly to be worth treating. She died of complications a few months later. “That’s what lit the fire,” he says.
From early on, Topaz was also a tech geek. After emigrating to Israel, where he served as an army medic, he helped implement its military’s first electronic health records system. As an undergrad at the University of Haifa, he focused on nursing informatics. And by the time he earned his PhD at the University of Pennsylvania, he was recognized as one of the field’s most promising young researchers. In 2018, after postdoctoral studies at Harvard Medical School and teaching appointments at Northeastern and Haifa, Topaz joined the faculty at Columbia Nursing.
Since then, he’s authored more than 240 peer-reviewed publications and received over $25 million in research grants. Partly through his efforts, Columbia Nursing was recently rated #1 in a global survey of AI nursing research published in the Journal of Nursing Management. A trailblazer in natural language processing, which uses machine learning to analyze text or speech, Topaz has created algorithms that can highlight references to wound care or comorbid conditions in nursing notes and help clinicians determine the urgency of drug allergy alerts. And working with VNS Health, he developed software that helps home care nurses decide how to prioritize patients upon admission based on a broad array of medical, cognitive, and sociodemographic risk factors.
In short, Topaz is in the vanguard of the AI revolution in healthcare, not in the skeptics’ circle. Recently, however, the rise of chatbots and other LLMs (also known as “generative AI”) has given him pause. “We’ve spent the past decade or so building analytic AI, testing it in clinical practice, and learning to mitigate its risks,” he says. “These LLMs, by contrast, have appeared in just the past two or three years, and they’ve been adopted with unprecedented speed.” Unlike earlier AI agents, he adds, they can run on any laptop, make decisions independently, and communicate fluently in any language or digital medium. “Those factors have made it crucial to assess their risks with increased urgency.”
Topaz, of course, is not alone in that assessment. As chatbots have shown their capacity for a wide range of mischief—from encouraging suicidal ideation in depressed teenagers to escaping a secure testing environment, banding together with other AI agents, and hacking into an outside database—even their inventors have begun to urge caution going forward. Microsoft cofounder Bill Gates, whose work helped make the AI revolution possible, recently warned in a blog post that the transition to the AI era “will be one of the most turbulent times in human history. Right now, we are not preparing adequately for that transition.”
Perhaps that’s why Topaz’s Lancet study made such a splash well beyond the world of nursing science. In an accompanying commentary, renowned pediatric researchers Howard Bauchner, MD, at Boston University’s School of Medicine, and Frederick Rivara, MD, at the University of Washington—former editors-in-chief, respectively, of the Journal of the American Medical Association (JAMA) and JAMA Pediatrics—congratulated Topaz and his coauthors for “shedding light on a very disturbing issue that threatens the integrity of research.”
More than 50 news outlets worldwide covered the paper in the days after it appeared, including Nature, STAT, CBS News, Fortune, The Scientist, and Retraction Watch. The study was also shared on social media by pundits with huge followings, including cardiologist and digital-medicine author Eric Topol, MD, and Nicholas Thompson, chief executive of The Atlantic.
The paper caught the attention of the editors of research journals as well. In June 2026, the Committee on Publication Ethics, which sets norms for journals worldwide, held a global forum on the topic titled “When Citation Integrity is Questioned.” Topaz’s findings were prominently referenced by the gathering’s moderators and participants. Around the same time, the preprint server arXiv announced that it would impose a one-year ban on authors who submit work containing unchecked AI-generated content, including fabricated references. The Lancet study was not mentioned in arXiv’s statement, but it was widely cited in the discussions it sparked.
“The paper helped open a public conversation that I hope will continue,” Topaz says. That dialogue has taken on a personal dimension for him, too, as strangers share their own stories about AI-hallucinated references. A geriatric researcher, for example, reported that she’d received a Google Scholar alert for a new paper that cited her, only to find that the referenced work was one she’d never written, attached to an identifier that led to an unrelated study. When the researcher wrote to the author, he doubled down and insisted that the cited paper was really hers. She had to contact the journal to get the citation removed.
In a way, Topaz observes, that researcher was lucky to have caught the fabrication. “She learned of it only because the fake carried her name,” he says. “Most of these references come with no such built-in alarm, allowing them to hide in plain sight.”
On an airplane flight, Topaz connected with another fan of his study. “The woman in the seat next to mine was doing something on her computer, and I asked if she was an academic,” he recalls. When she said she was, he asked if she ever used AI. “She started telling me about this article in The Lancet about fabricated papers. I told her, ‘I wrote that,’ and she was very surprised. She started showing me text messages about it that she’d written to her colleagues.”
The experience, Topaz says, was rare and humbling. “In my field, we look at problems that are critical to human health, but almost no one reads our papers. That was my Taylor Swift moment.”
Despite such recognition, Topaz has been frustrated since his study’s publication to see few changes in the way journals handle fabricated references. One reason may be that debate continues over what measures should be taken.
In the Lancet commentary, the authors argued that when a journal catches a fake citation, the only appropriate response is to retract the paper in which it appeared. Requiring already-overworked editors to determine the seriousness of the offense and an appropriate penalty is unfair, says former JAMA editor Bauchner. “Once you begin to retract the papers,” he asserts, “I guarantee you the behavior will change.”
Topaz, however, believes such problems should be dealt with on a case-by-case basis. Among papers found to contain fake citations, he points out, 91% had only one or two. In those cases, researchers may have tried to keep such references out, only to be outsmarted by a bot. At the other end of the scale are papers liberally studded with fabricated references, possibly due to the involvement of a “paper mill”—an organization that produces fake, plagiarized, or manipulated papers for researchers who pay to be listed as authors. “To me,” he says, “severity should be the indicator for adjudicating the case.”
He and his team are currently developing an automated system for determining the degree to which fake citations compromise a given paper, which could also be used to guide a calibrated response. “Sometimes, a fabricated reference is going to be central to the claim of the paper,” he explains. “In those cases, or with papers containing multiple fake references, a retraction may be justified. Where there’s just one fake, and its significance is peripheral, a simple correction may be enough.” These decisions, he adds, should be worked out “in collaboration between publishers and authors.”
Such debates are likely to become increasingly common as AI’s infiltration of healthcare research and practice gains momentum. That poses both challenges and opportunities for the field of nursing. At Columbia Nursing’s graduation ceremony this past May, Eric Horvitz, MD, PhD—Microsoft’s chief scientific officer and a seminal figure in the development of AI—gave the commencement address. (He also received the school’s Second Century Award for Excellence in Health Care.) “Much of my research has centered on what I call ‘AI in the open world,’” he told the crowd. “By that, I mean AI that must operate where you live and work, where information is incomplete, time is limited, and decisions are made under uncertainty. Nursing is the ultimate open-world discipline.”
The best AI systems, he continued, “will work with clinicians. They will complement human judgment, extend it, and improve through cycles of use, feedback, correction, refinement, and evaluation. And nurses must be central to those cycles.”
To Horvitz, Topaz’s study of fabricated references is a prime example of that kind of engagement. “We need to make sure that the credible core of scientific discourse remains bulletproof, and when we see a problem that will lead to distrust, we need to address it urgently,” he says. “Human oversight of these systems is critical.”
Topaz couldn’t agree more. “The instinct to ask, ‘But is anyone actually checking?’ is a nursing instinct,” he says. “It’s the question that drove this whole project.”