AI could flood genetics with convincing false findings, governance review warns

Generative artificial intelligence is being absorbed into genetics and genomics faster than the field’s governance mechanisms can absorb, and a new analysis argues that the resulting threat is not a matter of individual bad actors but of structural incentives. Writing in Frontiers in Genetics, Dov Greenbaum of Yale University, Reichman University, and Berkeley Law describes a systematic pressure on AI systems operating inside scientific publishing to optimize for publishability rather than truth, producing outputs that may be internally coherent yet empirically untethered. The article, a conceptual governance analysis rather than an empirical study, proposes embedding accountability mechanisms directly into the databases, repositories, journals, indexing platforms, and funding pipelines through which genomic knowledge flows, before recursive contamination becomes self-reinforcing.

The author is careful to frame the argument as a defense of AI rather than an attack on it. AI-driven variant classification and prioritization are increasingly used to accelerate rare-disease diagnosis, offering earlier answers to patients who might otherwise wait years, and computational methods are helping identify drug-repurposing candidates for rare diseases with limited commercial investment. The scale of modern genomic data has exceeded human analytical capacity, with population-scale gene-environment interactions, polygenic contributions, and epistatic effects across thousands of loci depending on computational methods no human team could replicate. The concern, Greenbaum writes, is that a tool this valuable, operating within an incentive structure already known to reward publishability over validity, requires governance proportional to both its promise and its risk.

The analysis traces three layers of integrity failure. The first is inadvertent contamination through hallucination. Unlike a fabricated legal citation, which can be checked against a closed, indexed corpus, a fabricated variant-phenotype association, spurious protein interaction, or invented effect size may carry plausible p-values, reasonable confidence intervals, and a biologically coherent mechanism, because the AI was trained on real biology and its fabrications inherit the surface features of real biology. The problem is already visible at scale: a Nature analysis reported that tens of thousands of 2025 publications may contain invalid AI-generated references, and one study of 13 large language models found hallucination rates of 14 to 95 percent, with invalid citation rates in 2.2 million citations rising 80.9 percent in 2025.

The second layer is deliberate fraud made cheaper and harder to detect. Publication-ready Western blots, flow cytometry plots, microscopy images, and histology slides no longer require a skilled forger, only a prompt, and AI industrializes paper mills by enabling synthetic manuscripts, fabricated datasets, and coherent but false reviews at volumes human fraud networks could never achieve. Forensic methods built for copy-paste artifacts and suspiciously clean data struggle against AI-generated fabrications trained to produce realistic noise and plausible variance. The third layer, epistemic blurring, is subtler: even accurate AI-assisted research can erode the transparency conditions that make science self-correcting, as analytical pathways become unreconstructable and the chain of custody that citation historically provided dissolves. Fabricated claims can at least be retracted once identified; lost attribution and provenance may be far harder to rebuild.

Underlying all three layers is what the article calls the reward function problem. Every AI system optimizes for its embedded objective, and in scientific systems the relevant reward signals include publication, citation, novelty, funding success, and downstream impact. An AI co-scientist trained on successful scientific outcomes will learn not the truth but the statistical and structural characteristics associated with work that gets accepted, cited, and funded. Human scientists possess a partial corrective, a conception of truth independent of institutional incentives, but AI systems have no analogous intrinsic brake unless such constraints are explicitly built into their objectives or architecture. The danger is not that any single AI-generated finding will be wrong, but that the ratio of empirically grounded claims to empirically untethered ones will shift so markedly that the cost of distinguishing reliable datapoints from unreliable ones will rise until the epistemic value of the infrastructure degrades for all users.

The feedback loop makes this dynamic worse. AI outputs optimized for publication enter databases and repositories, and subsequent systems train on that literature, learning the features of success with increasing precision. In genetics, the consequences are severe because the field relies on cumulative infrastructure: once contaminated outputs enter ClinVar or the GWAS Catalog, downstream annotation pipelines treat them as ground truth, and recurrence within the literature begins to substitute for independent confirmation. The article notes that among 166 gene-disease associations studied three or more times, only six replicated consistently, and more than 70 percent of 1,576 surveyed researchers reported failing to reproduce another’s experiments. A hallucinated association therefore enters a literature already saturated with non-replication and fragile claims, where a finding cited three times is not three times more likely to be true but three times harder to retract.

The article assesses science’s principal self-governance institutions and finds each structurally mismatched to AI-generated epistemic risk. Granting agencies shape conditions but cannot audit whether reviews contain hallucinated citations. Journals face peer review under siege, with only 24 percent of the 100 largest academic publishers having issued generative-AI guidance in an early systematic analysis, and AI-detection tools remaining unreliable and vulnerable to evasion. Prompt injection, the embedding of hidden instructions within a manuscript’s text or metadata, can influence an AI reviewer’s assessment without the human editor’s knowledge. Universities and professional societies govern through norms rather than enforcement, on timescales that generative AI does not respect. There is, the author emphasizes, no external regulator for truth claims: no licensing body for researchers as knowledge producers and no pre-market approval for a published finding.

The proposed response is infrastructure-embedded governance, sequenced by dependency. Near-term priorities include system-level retractability, in which every output from a discredited model version is flagged for review rather than just individual articles; public credibility ledgers that make an AI system’s auditable performance history visible to journals, funders, and databases; auditable computational lab notebooks that let users interrogate why a system produced a variant classification or drug-target prediction; and conflict-of-interest disclosure covering training-data provenance and commercial incentives. Trust-weighted indexing would incorporate credibility signals into search ranking so that articles with auditable provenance surface earlier, and reproducibility metrics would sit beside citation counts, distinguishing an article replicated three times from one cited three hundred times but never reproduced. Replication itself should be credited as first-class scholarship, the article argues, because trust cannot be policed into existence, only rewarded into existence.

Medium-term and continuing measures include sanctionability and exclusion mechanisms, bonding or insurance requirements for high-risk research AI, periodic revalidation of research AI systems, and controlled epistemic stress-testing, in which a consortium of curators, editors, and integrity researchers introduces deliberately fabricated but plausible test cases into controlled partitions of the scientific ecosystem to measure how quickly contamination propagates and which systems are most vulnerable. The article stresses that such test cases should never enter unrestricted scholarly databases, clinical systems, or public training corpora, and that testing should occur only in controlled environments capable of preventing uncontrolled downstream propagation. Most of the agenda, the author notes, requires no new technology or legislation, only institutional will and funder coordination, implementable through the existing authority of journals, funders, database curators, indexing services, and professional societies.

The window for preventive action, the analysis concludes, is finite. Once hallucinated findings enter shared databases, get cited by subsequent articles, and are ingested as training data by the next generation of AI systems, the contamination becomes recursive and self-reinforcing. The scientific record does not serve science alone; it underpins medicine, law, public health, and policy, and unreliable claims can propagate into clinical decisions, regulatory frameworks, and consequential public policies. The immediate goal is not to eliminate AI error but to prevent opaque error from becoming durable scientific infrastructure, and to realign scientific incentives so that trustworthiness is competitively rewarded before the gap between plausibility and verification becomes unbridgeable.

Subject of Research: Governance of generative AI's integrity risks in genetics and genomics publishing

Article Title: Scientific slop: generative AI, research incentives, and the integrity of the genomic knowledge base

Article References: Greenbaum, D. (2026). Scientific slop: generative AI, research incentives, and the integrity of the genomic knowledge base. Frontiers in Genetics, 17, Article 1894137. https://doi.org/10.3389/fgene.2026.1894137

Image Credits: AI Generated

DOI: 10.3389/fgene.2026.1894137

Keywords: generative AI, genomics, research integrity, scientific publishing, hallucination, peer review, retraction, reproducibility, research incentives, data provenance, AI governance, knowledge base contamination