• HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
Thursday, November 19, 2026
BIOENGINEER.ORG
No Result
View All Result
  • Login
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
No Result
View All Result
Bioengineer.org
No Result
View All Result
Home NEWS Science News Health

Providing Effect Sizes Fails to Eliminate Medical Student P-Value Misconceptions

by
November 19, 2026
in Health
Reading Time: 5 mins read
0
Providing Effect Sizes Fails to Eliminate Medical Student P-Value Misconceptions

Providing Effect Sizes Fails to Eliminate Medical Student P-Value Misconceptions

Share on FacebookShare on TwitterShare on LinkedinShare on RedditShare on Telegram

A recent study investigating the interpretive skills of 121 medical students from 24 German medical faculties demonstrates that the inclusion of descriptive statistics and standardized effect sizes in research abstracts is often insufficient to address widespread misconceptions regarding p-values and statistical inference. While reporting standardized effect sizes alongside p-values has long been recommended in biomedical research to support the interpretation of statistical results, the findings suggest that this practice alone does not systematically resolve fundamental misunderstandings. The research, which utilized fictitious but realistic medical abstracts, revealed that students frequently conflate statistical significance with clinical relevance, a tendency that remains persistent even when additional statistical data is explicitly provided. The persistence of these misconceptions across different stages of medical school underscores a critical challenge in medical statistics education, which may require a shift toward fostering a deeper conceptual understanding of statistical reasoning rather than relying solely on the inclusion of more information in summary reports.

To evaluate these interpretive patterns, researchers constructed a study using the digital learning environment CASUS, where each participant evaluated three distinct fictitious medical abstracts. These abstracts were carefully designed to vary systematically in their content and presentation of statistical evidence, allowing for a controlled examination of how different reporting formats influence interpretation. The study focused on two-tailed independent-samples t-tests comparing group means, a common statistical approach in medical research and a standard component of the German learning objectives catalogue for medicine, known as the NKLM. By holding the medical contexts consistent while manipulating the statistical presentation, the researchers sought to isolate the impact of different information sets on students’ abilities to evaluate the validity and meaning of the findings.

The study employed three distinct information sets to test how students responded to varying levels of statistical transparency. Information set 1 provided only the sample size and the p-value. Information set 2 expanded upon this by adding descriptive statistics, specifically the means and standard deviations for both the experimental and control groups. Finally, information set 3 included all of the above, supplemented by Cohen’s d, an established measure of the standardized mean difference between two groups. Participants were randomized in their evaluation of these abstracts across three different medical contexts—Hyperlipidemia, Cushing’s syndrome, and a study on fish oil—ensuring that each participant encountered each information set exactly once. Following their review of each abstract, students responded to a series of statements assessing their grasp of common misconceptions, including the relevance fallacy and the effect size fallacy.

The relevance fallacy occurs when statistical significance is incorrectly conflated with clinical or practical relevance, leading students to interpret statistically significant results as inherently meaningful and non-significant results as unimportant, regardless of the study’s specific medical context. In contrast, the effect size fallacy involves assuming that smaller p-values indicate larger observed effects. These misconceptions are conceptually distinct but both contribute to the over- or misinterpretation of statistical results. As the research team noted, clinical importance depends on contextual and substantive considerations, such as the minimal clinically important difference, rather than the dichotomous outcome of a p-value threshold. The study aimed to determine if providing additional data points—means, standard deviations, and Cohen’s d—would help students navigate these complexities more accurately.

Results from the analysis indicated that misconceptions were prevalent across all versions of the abstracts, regardless of the statistical information provided. On average, students endorsed between 1.76 and 2.85 of six specific statements reflecting common misconceptions. The researchers observed that for most contexts, there was no systematic difference in the number of endorsed misconceptions between the three information sets. This indicates that simply providing more statistical information in a research abstract is not a panacea for the persistent challenges in statistical literacy. The prevalence of these errors suggests that even when tools for interpreting effect size are provided, the foundational logic of statistical inference remains elusive for many students, potentially hindering their ability to apply research findings appropriately in clinical settings.

One notable exception to these general findings was observed in the context of a large sample size study involving a statistically significant result and a very small effect size. In this specific scenario, the provision of Cohen’s d appeared to reduce the tendency of students to endorse the relevance fallacy and the effect size fallacy. This suggests that additional statistical information may be most beneficial when the sample size is large enough to render very small, clinically negligible effects statistically significant. However, even in this more favorable context, students still struggled with several misconceptions, with an average of 2.02 of the six statements being incorrectly endorsed. The authors noted that an inconsistency in the textual description within this specific context may have introduced some level of confusion, potentially limiting the interpretability of this finding.

The study also examined how students’ estimation of effect size shifted based on the provided information, revealing that these estimates did not align well with the standardized mean differences expressed by Cohen’s d. Drawing on Tversky and Kahneman’s anchoring-and-adjusting heuristic, the researchers suggested that p-values served as an initial mental anchor for the students. Because they were conditioned to associate lower p-values with higher importance, students likely adjusted their perceived effect size estimates only to a limited extent, even when explicitly presented with contradictory data. This reliance on p-values may distract from a substantive interpretation of results, reinforcing the tendency to infer clinical magnitude from statistical significance, which remains a core problem in evidence-based practice.

Limitations of the study include the use of highly simplified, controlled scenarios that do not fully capture the complexity of real-world clinical research. The fictitious abstracts did not contain discussion or conclusion sections, which might have otherwise influenced participants’ interpretations. Additionally, the study found no association between the number of misconceptions endorsed and the participants’ semester of study, suggesting that current medical curricula may not be sufficiently fostering a growth in statistical literacy as students progress through their training. The lack of a correlation between academic progression and improved statistical reasoning indicates that persistent difficulties in interpreting inferential statistical results may endure throughout medical training unless targeted educational interventions are implemented.

These findings hold significant implications for both medical education and clinical practice. For educators, the results underscore the necessity of moving beyond the procedural application of inferential statistical methods to prioritize the development of statistical literacy and scientific reasoning skills. Approaches such as simulation-based learning, which engages students with the logic of random sampling and the distribution of expected differences under a null model, may provide a stronger conceptual foundation. By actively engaging with these logical processes, students may better grasp the nuances of interpreting results in varied clinical contexts, eventually leading to more informed and accurate decision-making.

In the clinical environment, the overreliance on p-values by clinicians—often under time constraints—can result in the misinterpretation of evidence, which may lead to inappropriate treatment preferences or miscommunications with patients regarding risks and uncertainties. The study concludes that while reporting effect sizes is essential, it must be accompanied by a broader educational focus on contextual evaluation. The use of alternative reporting measures, such as the number needed to treat, may further assist clinicians in contextualizing findings. Ultimately, the transition toward a more nuanced appraisal of statistical evidence, coupled with a focus on clinically important differences rather than dichotomous significance, is essential for advancing evidence-based medicine and ensuring the quality of patient care.

Subject of Research: The study investigated the effect of presenting descriptive statistics and effect sizes in medical abstracts on students' p-value misconceptions.

Article Title: Not a panacea: the role of presenting descriptive statistics and effect sizes in abstracts on medical students’ misconceptions about p-values

Article References: Eisenhut, F. S., Wiemann, L., Binder, K., Fischer, M. R., Schmidmaier, R., & Braun, L. T. (2026). Not a panacea: the role of presenting descriptive statistics and effect sizes in abstracts on medical students’ misconceptions about p-values. BMC Medical Research Methodology, 26(1), Article 253. https://doi.org/10.1186/s12874-026-03075-w

Image Credits: AI Generated

DOI: 10.1186/s12874-026-03075-w

Keywords: Medical Education, Statistical Literacy, P-Values, Effect Sizes, Scientific Reasoning, Medical Research Methodology, Clinical Decision Making, Statistical Inference, Cohen's D, Significance Testing, Relevance Fallacy, Effect Size Fallacy, Research Abstracts, Evidence-Based Medicine, German Medical Faculties

News Source: Ophelia Keating. (November 19, 2026). Providing Effect Sizes Fails to Eliminate Medical Student P-Value Misconceptions. Scienmag.

Tags: Clinical Decision-MakingCohen's dEffect Size Fallacyeffect sizesEvidence-based medicineGerman Medical FacultiesMedical Educationmedical research methodologyp-valuesRelevance FallacyResearch Abstractsscientific reasoningSignificance Testingstatistical inferenceStatistical Literacy
Share12Tweet7Share2ShareShareShare1

Related Posts

Perceived School Support Linked to Digital Civic Responsibility in Chinese Vocational Students

Perceived School Support Linked to Digital Civic Responsibility in Chinese Vocational Students

November 19, 2026
Porcine Epidemic Diarrhea Virus Demonstrates Persistent Coronavirus Evolution in Managed Livestock

Porcine Epidemic Diarrhea Virus Demonstrates Persistent Coronavirus Evolution in Managed Livestock

November 19, 2026

Hypertensive Patients on ARBs Show Hemodynamic Reductions During Prone Spine Surgery

November 19, 2026

Tissue HLA-G Positivity Linked to Deep Myometrial Invasion in Macanese Endometrioid Cancer

November 19, 2026

POPULAR NEWS

  • First-Principles Calculations Reveal High Thermoelectric Efficiency and Half-Metallicity in Co2VPb Alloy

    First-Principles Calculations Reveal High Thermoelectric Efficiency and Half-Metallicity in Co2VPb Alloy

    29 shares
    Share 12 Tweet 7
  • Mobile Bird Hosts Enable Regional Gene Flow in Crater Lake Parasites

    29 shares
    Share 12 Tweet 7
  • Ultrasound Shows Low Diagnostic Accuracy for Sexing Immature Agile Frogs

    29 shares
    Share 12 Tweet 7
  • Lithuanian Forest Foragers Link Solitary Mushroom Gathering to Cross-Border Trade

    29 shares
    Share 12 Tweet 7

About

We bring you the latest biotechnology news from best research centers and universities around the world. Check our website.

Follow us

Recent News

Perceived School Support Linked to Digital Civic Responsibility in Chinese Vocational Students

Porcine Epidemic Diarrhea Virus Demonstrates Persistent Coronavirus Evolution in Managed Livestock

Hypertensive Patients on ARBs Show Hemodynamic Reductions During Prone Spine Surgery

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 88 other subscribers
  • Contact Us

Bioengineer.org © Copyright 2023 All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Homepages
    • Home Page 1
    • Home Page 2
  • News
  • National
  • Business
  • Health
  • Lifestyle
  • Science

Bioengineer.org © Copyright 2023 All Rights Reserved.