• HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
Thursday, November 19, 2026
BIOENGINEER.ORG
No Result
View All Result
  • Login
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
No Result
View All Result
Bioengineer.org
No Result
View All Result
Home NEWS Science News Health

ChatGPT Assessment in Radiomics Reflects Human Concordance Rather Than Internal Repeatability

by
November 13, 2026
in Health
Reading Time: 3 mins read
0
ChatGPT Assessment in Radiomics Reflects Human Concordance Rather Than Internal Repeatability

ChatGPT Assessment in Radiomics Reflects Human Concordance Rather Than Internal Repeatability

Share on FacebookShare on TwitterShare on LinkedinShare on RedditShare on Telegram

Researchers investigating large language model applications in radiomics quality assessment have clarified that their evaluation measured agreement between artificial intelligence and human experts rather than within-model repeatability. Writing in European Radiology Experimental, the author team addressed methodological distinctions highlighted by Yang, Meng, and Shen regarding how artificial intelligence tools are validated in medical imaging research. The correspondence emphasized that reported intraclass correlation coefficients reflect concordance between model outputs and expert scoring under a specific evaluation configuration, rather than demonstrating consistent performance across repeated evaluations.

In the primary investigation, the investigators evaluated the performance of ChatGPT 5.1 Thinking in applying two standardized radiomics evaluation tools, known as METRICS and the Radiomics Quality Score (RQS). The model evaluated each research article once using standardized prompts in a single-pass framework. To prevent carry-over effects between evaluations, each prompt was deployed in a separate temporary session with the platform memory feature disabled. Because articles were not assessed multiple times, the study was not designed to retrospectively quantify run-to-run variability or comprehensively evaluate internal model reliability.

The authors also clarified the structure of the human expert evaluations used as the comparator in the study. The primary benchmark was established by two human readers with complementary domain expertise who independently scored items within their specific fields. These item-level scores were subsequently combined into a single composite assessment, a method that was likewise applied across secondary reader groups A and B without holding consensus meetings. Consequently, the statistical comparisons evaluated ChatGPT against composite expert evaluations rather than against individual human readers scoring the full set of METRICS and RQS criteria.

Addressing the statistical specifications of the study, the investigators explained their selection of an absolute-agreement, single-measure formulation for calculating intraclass correlation coefficients. This analytical structure treated systematic differences between scores as disagreement and relied on a single score from each assessment setup rather than an averaged score across multiple sessions. The authors noted that because human reader groups, model choice, and prompting strategies were fixed, the resulting coefficients reflect agreement between those specific configurations rather than estimates that can be broadly generalized to other readers, alternative language models, or different prompting strategies.

Technical constraints associated with the testing environment also shaped the scope of the findings. The investigators accessed ChatGPT 5.1 Thinking through its web application interface on December 8, 2025, where sampling parameters such as temperature were not user-configurable. To advance reproducibility and distinguish random stochastic variability from systematic scoring discrepancies, the authors recommended that future validation studies preserve run-level outputs across repeated assessments and document model parameters whenever platform access permits.

The author team concluded that item-level discrepancies identified in their study represent descriptive patterns of single-pass assessments rather than stable, persistent biases across repeated model executions. While highlighting that repeated assessments will be necessary to evaluate intra-model repeatability in future work, the researchers maintained their initial conclusion: ChatGPT 5.1 Thinking shows potential to assist in radiomics quality scoring, but ongoing discrepancies with composite expert evaluations continue to warrant caution.

Regarding the mathematical modeling choices, the researchers explained that their intraclass correlation coefficient calculations yield identical numerical results under alternative statistical formulations. Specifically, when applied to their collected data, a two-way mixed-effects, absolute-agreement, single-measure model produces the exact same numerical point estimate as a two-way random-effects formulation. The author team emphasized that selecting between these two formulations affects the theoretical interpretation of generalizability rather than altering the reported numerical concordance values.

Subject of Research: Evaluation of large language model repeatability and agreement in radiomics quality assessment

Article Title: Reply to the Letter to the Editor: Separating LLM repeatability from expert concordance in radiomics quality assessment

Article References: Garello, L. F., Giannini, V., Gatti, M., Defeudis, A., Cafaro, D., Nicoletti, G., Culasso, N. C., Veltri, A., Cuocolo, R., & Balbi, M. (2026). Reply to the Letter to the Editor: Separating LLM repeatability from expert concordance in radiomics quality assessment. European Radiology Experimental, 10(1), Article 180. https://doi.org/10.1186/s41747-026-00859-z

Image Credits: AI Generated

DOI: 10.1186/s41747-026-00859-z

Keywords: radiomics quality assessment, ChatGPT 5.1 Thinking, intraclass correlation coefficient, METRICS score, Radiomics Quality Score, intra-model repeatability, human-LLM agreement, single-pass evaluation, composite expert assessment, large language models, two-way mixed-effects model, absolute agreement, model inference conditions, run-to-run variability, European Radiology Experimental, stochastic variability, medical imaging quality, standardized prompting

News Source: Ophelia Keating. (November 13, 2026). ChatGPT Assessment in Radiomics Reflects Human Concordance Rather Than Internal Repeatability. Scienmag.

Tags: absolute agreementChatGPT 5.1 Thinkingcomposite expert assessmentEuropean Radiology Experimentalhuman-LLM agreementintra-model repeatabilityintraclass correlation coefficientLarge Language Modelsmedical imaging qualityMETRICS scoremodel inference conditionsradiomics quality assessmentRadiomics Quality Scorerun-to-run variabilitysingle-pass evaluationstandardized promptingstochastic variabilitytwo-way mixed-effects model
Share12Tweet7Share2ShareShareShare1

Related Posts

Cardiac MRI and Ultrasound Strain Measures Differ in Chronic Ischemic Disease

Cardiac MRI and Ultrasound Strain Measures Differ in Chronic Ischemic Disease

November 19, 2026
Mathematical Formulas for Ventilator Energy Show Wide Discrepancies in Intensive Care

Mathematical Formulas for Ventilator Energy Show Wide Discrepancies in Intensive Care

November 19, 2026

Network Meta-Analysis Compares Janus Kinase Inhibitors and Adalimumab in Psoriatic Arthritis

November 19, 2026

Aging Drives Rising Absolute Neurological Disorder Burden Across Western Pacific Nations

November 19, 2026

POPULAR NEWS

  • East Hararghe Farmers Show Mixed Perceptions Toward Irrigated Cluster Wheat Farming

    East Hararghe Farmers Show Mixed Perceptions Toward Irrigated Cluster Wheat Farming

    29 shares
    Share 12 Tweet 7
  • Fruit and Vegetable Consumption Declined Among Iranian Adults With Hypertension Over 15 Years

    29 shares
    Share 12 Tweet 7
  • Multiscale Fusion Dehazing Algorithm Preserves Sharp Edges in Foggy Driving Vision

    29 shares
    Share 12 Tweet 7
  • Periorbital Anthrax Mimicked Necrotizing Infection Following Soil Trauma in Case Report

    29 shares
    Share 12 Tweet 7

About

We bring you the latest biotechnology news from best research centers and universities around the world. Check our website.

Follow us

Recent News

Cardiac MRI and Ultrasound Strain Measures Differ in Chronic Ischemic Disease

Mathematical Formulas for Ventilator Energy Show Wide Discrepancies in Intensive Care

Network Meta-Analysis Compares Janus Kinase Inhibitors and Adalimumab in Psoriatic Arthritis

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 89 other subscribers
  • Contact Us

Bioengineer.org © Copyright 2023 All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Homepages
    • Home Page 1
    • Home Page 2
  • News
  • National
  • Business
  • Health
  • Lifestyle
  • Science

Bioengineer.org © Copyright 2023 All Rights Reserved.