• HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
Monday, November 16, 2026
BIOENGINEER.ORG
No Result
View All Result
  • Login
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
No Result
View All Result
Bioengineer.org
No Result
View All Result
Home NEWS Science News Health

AI Models Show Inconsistent Performance and Safety Risks in Pregnancy Asthma Advice

by
November 11, 2026
in Health
Reading Time: 3 mins read
0
AI Models Show Inconsistent Performance and Safety Risks in Pregnancy Asthma Advice

AI Models Show Inconsistent Performance and Safety Risks in Pregnancy Asthma Advice

Share on FacebookShare on TwitterShare on LinkedinShare on RedditShare on Telegram

Large language models evaluated on patient inquiries regarding asthma during pregnancy demonstrated uneven clinical quality and notable safety risks, with nearly one-third of generated answers containing safety concerns despite high marks for certain systems. In a cross-sectional comparative study conducted by investigators from Charité University Medicine Berlin and Zhejiang Hospital, researchers assessed how current artificial intelligence platforms handle high-risk reproductive health topics. The findings revealed that even when artificial intelligence applications produce answers judged favorably by clinical raters, their outputs frequently introduce clinical hazards or communication barriers that preclude unsupervised patient deployment.

Patients increasingly turn to artificial intelligence interfaces to interpret medical conditions and treatment choices, yet the suitability of these systems for sensitive reproductive medicine remains uncertain.

To establish a representative and standardized curriculum of patient-facing inquiries, the research team implemented a two-stage modified Delphi-informed consensus process. The selection procedure began with an initial pool of 40 candidate questions reflecting common dilemmas and management challenges surrounding asthma during pregnancy. Through iterative expert review, the evaluators narrowed the pool to 30 candidates before finalizing 19 retained questions specifically structured to test the breadth and depth of model knowledge across typical patient scenarios.

The experimental protocol tested six distinct large language models, prompting each system once per retained question under a standardized structured prompt. This testing matrix generated 114 distinct model responses and 342 rater-level evaluation records for detailed comparative analysis.

Three clinical evaluators independently examined the resulting outputs across several validated assessment instruments. The evaluators scored overall clinical response quality alongside information quality using the standardized DISCERN instrument, which evaluates health information reliability and treatment descriptions. In addition, the raters scored understandability and actionability using the Patient Education Materials Assessment Tool, known as PEMAT, while computational analyses calculated four established readability indices to determine the structural complexity of the generated text.

Statistical evaluation using matched question-level nonparametric tests with multiplicity correction and effect sizes revealed significant divergence in clinician-rated overall clinical response quality across the six evaluated architectures. The Friedman test indicated substantial variation among the systems (Friedman chi-square [5, N = 19] = 72.31, P < 0.001; Kendall’s W = 0.76). While GPT-5.5 and Claude Sonnet 5 achieved the highest observed median quality scores among the cohort, statistical testing demonstrated no significant difference between the two top-performing models after applying multiplicity corrections.

Beyond overall clinical response scores, performance across secondary metrics proved highly fragmented among the competing systems. Model rankings fluctuated substantially across DISCERN information quality, PEMAT understandability, PEMAT actionability, and automated readability calculations, with no single model demonstrating consistent leadership across all evaluated outcome measures. Crucially, none of the six evaluated language models achieved a median Flesch-Kincaid grade level at or below grade 8, indicating that all tested systems generated text too complex for general patient education standards.

Patient safety emerged as a primary concern during the investigation’s dedicated revision-stage safety analysis. Two clinicians independently reviewed all 114 responses using a predefined, structured safety taxonomy, demonstrating an inter-rater agreement of 87.7 percent for the presence of any safety concern (Cohen’s kappa = 0.70). Any response marked by rater disagreement or prespecified severity triggers underwent blinded adjudication by a third senior clinician, who ultimately confirmed that 37 of the 114 responses (32.5 percent) contained at least one safety concern, including 4 responses (3.5 percent) classified as high-risk unsafe advice.

The study authors concluded that while large language models show promising capabilities under structured prompting conditions, their performance remains fundamentally uneven across maternal asthma inquiries. High clinician-rated overall clinical response quality did not guarantee superior information quality, usability, appropriate reading levels, or clinical safety. Because the findings fail to demonstrate that current artificial intelligence models can be safely utilized without supervision, the authors emphasized that all model-generated patient guidance must undergo guideline-based clinical verification, explicit safety adjudication, and professional readability editing before clinical implementation.

Subject of Research: Multidimensional evaluation of large language models for patient education on asthma during pregnancy

Article Title: Multidimensional evaluation of large language models for patient education on asthma during pregnancy: a cross-sectional comparative study

Article References: Zhu, Z., Xie, C., & Wu, H. (2026). Multidimensional evaluation of large language models for patient education on asthma during pregnancy: a cross-sectional comparative study. Frontiers in Digital Health, 8, Article 1958383. https://doi.org/10.3389/fdgth.2026.1958383

Image Credits: AI Generated

DOI: 10.3389/fdgth.2026.1958383

Keywords: large language models, asthma during pregnancy, patient education, reproductive medicine, artificial intelligence in medicine, digital health, clinical response quality, DISCERN, PEMAT, readability indices, Flesch-Kincaid grade level, patient safety, clinical adjudication, cross-sectional study, Delphi method, health informatics, maternal health, medical advice safety

News Source: Harold Sullivan. (November 11, 2026). AI Models Show Inconsistent Performance and Safety Risks in Pregnancy Asthma Advice. Scienmag.

Tags: Artificial intelligence in medicineasthma during pregnancyclinical adjudicationclinical response qualityCross-sectional StudyDelphi methodDigital HealthDISCERNFlesch-Kincaid grade levelhealth informaticsLarge Language ModelsMaternal Healthmedical advice safetyPatient EducationPatient SafetyPEMATreadability indicesReproductive medicine
Share12Tweet7Share2ShareShareShare1

Related Posts

Interferon-Induced LAP3 Promotes Mouse Embryo Resorption by Inhibiting Trophoblast Mitophagy

Interferon-Induced LAP3 Promotes Mouse Embryo Resorption by Inhibiting Trophoblast Mitophagy

November 16, 2026
Biliary Fascioliasis Mimics Bile Duct Stones: Insights from Four Clinical Cases

Biliary Fascioliasis Mimics Bile Duct Stones: Insights from Four Clinical Cases

November 16, 2026

Physical Fitness Declined Among Belgian Children Between 2004 and 2022

November 16, 2026

AlphaFold3 Outperforms Conventional Tools in Predicting Adeno-Associated Virus Capsid Structures

November 16, 2026

POPULAR NEWS

  • Urban Expansion Across Sub-Saharan Africa Fuels Complex Malaria Transmission Risks

    Urban Expansion Across Sub-Saharan Africa Fuels Complex Malaria Transmission Risks

    29 shares
    Share 12 Tweet 7
  • Distinct Nesprin Isoforms Govern Nuclear Positioning and Mechanics in Striated Muscle

    29 shares
    Share 12 Tweet 7
  • Language Model Simulation Links Control Prompts to Synthetic Quiet Quitting Risk

    29 shares
    Share 12 Tweet 7
  • Standardized Metrics Needed to Prioritize Own Mother’s Milk in Neonatal Units

    29 shares
    Share 12 Tweet 7

About

We bring you the latest biotechnology news from best research centers and universities around the world. Check our website.

Follow us

Recent News

Interferon-Induced LAP3 Promotes Mouse Embryo Resorption by Inhibiting Trophoblast Mitophagy

Systematic Review Outlines Artificial Intelligence Strategies for Designing Functional Taste Peptides

Biliary Fascioliasis Mimics Bile Duct Stones: Insights from Four Clinical Cases

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 88 other subscribers
  • Contact Us

Bioengineer.org © Copyright 2023 All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Homepages
    • Home Page 1
    • Home Page 2
  • News
  • National
  • Business
  • Health
  • Lifestyle
  • Science

Bioengineer.org © Copyright 2023 All Rights Reserved.