A debate over whether modern science and technology are becoming less disruptive has entered a new phase, with researchers arguing that the apparent decline may be real—but that the way scholarly and patent databases are built could be influencing how strongly that decline appears. In a reply published in Nature, Matthew Park, Erin Leahey and Russell J. Funk respond to the critique “Dataset artefacts can partially drive the measured decline in disruption,” defending the broader interpretation of disruption trends while acknowledging that imperfections in research datasets can affect the measurement. Their response addresses a question with consequences far beyond academic statistics: is innovation genuinely becoming less transformative, or are today’s databases failing to recognize the breakthroughs that matter?
The controversy centers on the concept of “disruption,” a term increasingly used to describe scientific papers and inventions that redirect the trajectory of a field. A highly disruptive paper may cause later work to rely less on earlier studies, suggesting that it has opened a new intellectual path rather than merely extending an existing one. In patent analysis, a disruptive invention can similarly make previous technologies less central by introducing a novel approach. Researchers attempt to quantify this effect using citation and reference patterns, transforming millions of records into numerical indicators that estimate whether a contribution consolidates existing knowledge or pushes researchers toward a different direction.
These measurements are powerful because they allow scholars to study innovation across decades, disciplines and countries. They are also vulnerable to the structure of the data. Citation databases do not capture every publication equally, and their coverage can change as journals, conference proceedings, patents and digital archives are added or removed. Older records may contain incomplete reference lists, inconsistent author names or missing links between related documents. Newer records can be indexed more comprehensively, while the growth of collaborative research may alter how papers cite one another. Each of these factors can influence the mathematical calculation of disruption, even when the underlying scientific activity has not changed.
The critique addressed by Park, Leahey and Funk argues that such dataset artefacts can partially contribute to the measured decline in disruption. In practical terms, the concern is that an apparent historical trend might reflect changes in the observational instrument rather than a complete transformation of innovation itself. If the data contain more comprehensive references for recent papers, for example, those papers may appear more firmly connected to prior work. A metric that interprets dense backward citation as evidence of consolidation could then assign lower disruption scores to newer research. The resulting pattern would resemble a decline in breakthrough activity, even if some of the difference arose from improved record-keeping.
The authors’ reply is important because it places a boundary around that criticism. The existence of measurement bias does not automatically invalidate the phenomenon being measured. In empirical science, a dataset can be imperfect while still revealing a meaningful signal, provided researchers test how sensitive their conclusions are to known weaknesses. The central issue is therefore not whether artefacts exist—most large historical datasets contain them—but whether correcting for those artefacts eliminates the observed decline or merely reduces its size. Park, Leahey and Funk’s response maintains that the critique should be considered in interpreting the results, while arguing that it does not by itself explain away the broader pattern.
This distinction matters technically. A trend can be decomposed into at least two components: a change in the real-world process and a change in the measurement system. If the disruption score for a paper depends on who cites it, which references are recorded and how the database links documents, then variations in indexing quality can propagate through the metric. Researchers can investigate this by comparing multiple databases, restricting analyses to periods with more stable coverage, testing alternative definitions of disruption and modeling the effects of missing or incorrectly recorded references. Robust findings should persist, at least in direction, when reasonable analytical choices are changed. Findings that disappear under modest corrections would require a much more cautious interpretation.
The exchange also highlights a deeper problem in evaluating innovation. Disruption is not identical to originality, usefulness or social impact. A paper may introduce a technically novel idea but attract few citations because it is difficult to apply, published in a poorly indexed venue or relevant to a small community. Conversely, a widely cited contribution may organize and strengthen an existing field without overturning it. Citation-based indicators are therefore indirect measures: they infer intellectual influence from patterns of attention rather than observing novelty directly. The reply reinforces the need to treat these scores as evidence about the evolution of knowledge, not as definitive labels attached to individual discoveries.
For science policy, the stakes are high. Claims that research is becoming less disruptive have been used to explain concerns about stagnant productivity, rising publication volumes and the increasing difficulty of producing major advances. Universities and funding agencies may respond by changing evaluation systems, rewarding riskier projects or encouraging researchers to work across disciplinary boundaries. But if part of the decline is produced by data construction, policies based on an unqualified reading of the trend could misdiagnose the problem. At the same time, dismissing the decline entirely because databases are imperfect could be equally misleading. The reply supports a middle position: improve the measurements, quantify uncertainty and avoid turning a complex statistical pattern into a simple verdict about the state of human creativity.
The debate is likely to intensify as artificial intelligence, automated laboratories and rapidly expanding digital archives reshape the research ecosystem. New tools may accelerate discovery, but they may also produce larger teams, more incremental publications and citation networks that are harder to interpret. Machine-generated text, preprints, software, datasets and nontraditional research outputs further complicate the definition of a scientific contribution and the task of tracking its influence. Park, Leahey and Funk’s response therefore speaks to a broader challenge facing quantitative science studies: before asking whether innovation is slowing, researchers must establish how reliably innovation can be seen through the data. The decline in measured disruption may be partly technological, partly institutional and partly statistical—but determining the balance will require better records, more transparent methods and indicators capable of recognizing breakthroughs that conventional citations overlook.
Subject of Research: The measurement of scientific and technological disruption, and the influence of dataset artefacts on observed trends in innovation.
Article Title: Reply to: Dataset artefacts can partially drive the measured decline in disruption.
Article References: Park, M., Leahey, E. & Funk, R. J. “Reply to: Dataset artefacts can partially drive the measured decline in disruption.” Nature 656, E14–E21 (2026). https://doi.org/10.1038/s41586-026-10788-x
Image Credits: AI Generated
DOI: 10.1038/s41586-026-10788-x
Keywords: Scientific disruption, technological innovation, citation analysis, bibliometrics, research databases, dataset artefacts, innovation measurement, science policy
Tags: Challenges in quantifying scientific disruptionCitation analysis in measuring research impactDebate on declining innovation disruptivenessDisruption measurement in scientific researchImpact of database artifacts on innovation analysisInfluence of dataset imperfections on disruption trendsLimitations of current datasets in innovation researchLong-term implications of disruption measurementMethodological considerations in disruption studiesPatent analysis for technological disruptionRole of scholarly and patent databases in innovation metricsTrends in technological disruption and scientific breakthroughs
