An editorial published in the journal Ecology and Evolution is urging researchers, peer reviewers, and readers across ecological and evolutionary disciplines to reform their interpretation of statistical testing by moving away from arbitrary significance thresholds. The authors observe that ecology and evolutionary biology operate largely as empirical and quantitative fields grounded in theoretical and practical foundations established by earlier mathematicians and statisticians. The prevailing convention of statistical significance, originally introduced by R. A. Fisher and subsequently expounded by Jerzy Neyman and Egon Pearson, has become deeply enculturated in ecological modeling. Under this traditional framework, an alpha threshold of p equals 0.05 serves as a rigid arbiter that splits analytical outcomes into binary designations of true or false, or significant versus insignificant, despite long-standing recognition that such hardline categorizations are fundamentally problematic.
The critique of null-hypothesis testing in wildlife sciences gained substantial momentum following Douglas Johnson’s 1999 publication on the insignificance of statistical significance testing, which initiated a broader transition toward more robust analytical methods. That work prompted a widespread shift where traditional null-hypothesis testing was largely replaced by multiple model selection rooted in information theory, as formulated by Hirotugu Akaike and expanded by Kenneth Burnham and David Anderson. Nevertheless, p-values remain prevalent in contemporary scientific practice as tools for evaluating how an observed sample mean deviates from a second sample or a theoretical population mean. While acknowledging that p-values retain practical utility when appropriately applied, the editorial emphasizes that authors must ensure their analytical reporting remains strictly free of misinterpretation, drawing attention to a wider body of literature that addresses persistent statistical fallacies.
To guide researchers toward sounder inference, the editorial highlights foundational principles established in the American Statistical Association statement on p-values formulated by Ronald Wasserstein and Nicole Lazar. The statement clarifies that p-values can indicate how incompatible observed data are with a specified statistical model, but they do not measure the probability that a studied hypothesis is true, nor do they reflect the probability that data were produced solely by random chance. Crucially, scientific conclusions as well as policy or business decisions should not be based solely on whether a calculated p-value crosses a specific numerical boundary. The editorial underscores that there is no mathematical or logical foundation for establishing a hardline demarcation between truth and falsehood, meaning that valid conclusions require researchers to weigh probabilities thoughtfully and carefully using comprehensive analytical context.
Transparent scientific evaluation requires full reporting of all statistical output rather than selective disclosure of isolated results that cross significance thresholds. The editorial advises researchers to avoid presenting only significant p-values, urging them instead to report all calculated p-values alongside corresponding effect sizes, sample sizes, and attendant error estimates so that readers can evaluate the evidence independently. A fundamental tenet of proper interpretation is that a p-value does not measure the magnitude of an effect or the scientific importance of a result. Statistical significance cannot be equated with scientific significance or practical impact; for example, a regression yielding a slope of 0.01 with a p-value of 0.002 does not reflect a stronger relationship than a regression yielding a slope of 1.0 with a p-value of 0.07, where the effect size is one hundred times greater despite a lower probability of departure from the assumed distribution.
When evaluated in isolation, a p-value fails to provide an adequate measure of evidence regarding a specific model or hypothesis. Addressing this limitation requires integrating information theory, model validation, and supplementary diagnostic tools to provide meaningful insight into the empirical support underlying an analytical framework. As Sander Greenland and colleagues elaborated, standard introductory life-science curricula commonly teach that p equals 0.05 serves as an arbiter of truth, obscuring what the statistic actually reflects. In practice, the distance between observed data and model predictions is quantified through a test statistic, such as a t-statistic or a Chi-squared statistic. The p-value represents the probability that the chosen test statistic would be at least as large as the observed quantity if every underlying model assumption, including the target test hypothesis, were entirely correct.
This technical definition reveals that p-values evaluate the entire constellation of assumptions underlying data generation rather than evaluating the targeted hypothesis in isolation. Although a small p-value denotes a marked departure from the assumed data distribution, it cannot isolate where that departure originated, whether in the underlying effect size, sample size, study design, structural model specification, or mathematical assumption violations. Consequently, a p-value provides no definitive information about a hypothesis unless an investigator is entirely certain that all other modeling assumptions have been fully and repeatably satisfied. Even under ideal conditions, a p-value functions strictly as a continuous gauge of compatibility between the data and the entire analytical model, scaling continuously from zero for complete incompatibility to one for perfect compatibility, a reality recently reinforced in Ecology and Evolution by Arno and colleagues.
Statistical misconceptions within ecology and evolution extend beyond p-values to encompass the widespread misuse of confidence intervals. Investigators frequently interpret overlapping confidence intervals between two sample means as proof that the underlying populations do not differ, or treat an interval overlapping zero as definitive evidence of no effect, both of which represent flawed assumptions. Drawing on Greenland and colleagues, the editorial identifies several specific false interpretations: assuming a calculated ninety-five percent confidence interval has a ninety-five percent probability of containing the true effect size; believing that an effect size lying outside the interval has been refuted by the data; concluding that differences are non-significant because intervals overlap; claiming that an observed interval predicts that ninety-five percent of future study estimates will fall within it; or assuming an interval excluding the null is inherently more precise than one including it.
Compounding these misinterpretations, professional practice has been remarkably slow to absorb reform guidance despite high-profile interventions across the scientific community. A bibliographic search within Wiley journals revealed 867,449 occurrences of the phrase statistically significant since 2020, with 8,124 appearances documented in Ecology and Evolution alone. Studies by Berner and Amrhein demonstrated extensive misuse of p-values in Evolutionary Biology, while recent instructional articles, such as Davis and Kay’s guide to writing statistical methods, continue to advocate for strict cutoffs like p equals 0.05. Furthermore, reviews by Matthews and colleagues one year following the American Statistical Association statement documented minimal changes in reporting habits, echoing sentiments by Hurlbert and colleagues and the collective call by eight hundred signatories in Nature led by Valentin Amrhein to retire statistical significance entirely.
To navigate research communication without reliance on significance thresholds, the editorial points to recommendations formulated by Ronald Wasserstein and colleagues under the acronym ATOM, which instructs scientists to accept uncertainty, be thoughtful, open, and modest. Authors are encouraged to focus their presentations and discussions primarily on effect sizes and attendant error estimates while rejecting arbitrary alpha values or overlapping interval rules as metrics of analytical quality. In place of binary pronouncements, manuscripts should articulate the strength of evidence and disclose complete sample sizes, effect sizes, and exact p-values, allowing the reader to judge. Methodological rigor also requires explicit reporting of all model assumptions, candid evaluations of potential violations, and the systematic use of sensitivity testing to examine whether findings remain robust across alternative analytical decisions.
Ultimately, statistical models represent imperfect yet useful instruments for investigating natural phenomena, aligning with George Box’s aphorism that all models are wrong but some are useful. Because the utility of any statistical tool depends on the appropriateness and skill of its practical application, the editorial emphasizes that scientific insight cannot emerge from rigid binary demarcations between significant and non-significant results. Instead, sound ecological conclusions require quantitative and qualitative synthesis across multiple streams of empirical evidence. By encouraging authors and reviewers to adopt transparent reporting practices, examine effect sizes, and critically assess model assumptions, the journal aims to advance a more informative, robust, and meaningful standard for communicating ecological and evolutionary discoveries.
Subject of Research: Statistical interpretation and reporting practices in ecology and evolutionary biology
Article Title: A “Significant” Shift in Statistical Interpretation in Ecology and Evolution: A Helpful Guide for Authors and Reviewers
Article References: Fisher, J. T. (2026). A “Significant” Shift in Statistical Interpretation in Ecology and Evolution : A Helpful Guide for Authors and Reviewers. Ecology and Evolution, 16(11), Article e74659. https://doi.org/10.1002/ece3.74659
Image Credits: AI Generated
DOI: 10.1002/ece3.74659
Keywords: statistical significance, p-values, confidence intervals, ecology, evolutionary biology, null-hypothesis testing, effect sizes, model selection, information theory, American Statistical Association, data transparency, model assumptions, sensitivity testing, scientific inference, research methodology, statistical interpretation, ecological modeling, error estimation
News Source: Gavin Prescott. (November 12, 2026). Ecology and Evolution Editorial Urges Researchers to Abandon Arbitrary Statistical Thresholds. Scienmag.



