• HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
Monday, November 16, 2026
BIOENGINEER.ORG
No Result
View All Result
  • Login
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
No Result
View All Result
Bioengineer.org
No Result
View All Result
Home NEWS Science News Health

Six AI Chatbots Show Variable Performance Answering Diabetic Kidney Disease Questions

by
November 8, 2026
in Health
Reading Time: 3 mins read
0
Six AI Chatbots Show Variable Performance Answering Diabetic Kidney Disease Questions

Six AI Chatbots Show Variable Performance Answering Diabetic Kidney Disease Questions

Share on FacebookShare on TwitterShare on LinkedinShare on RedditShare on Telegram

In a cross-sectional comparative study guided by the CHART framework, researchers found that publicly accessible large language models can provide plausible information about diabetic kidney disease but demonstrate notable variability in performance and clinical safety risks. The investigation revealed that safe-response rates across six evaluated chatbots ranged between 83.3 percent and 92.9 percent, with no statistically significant differences in overall safety among the systems. However, approximately 11.9 percent of all generated answers were classified as potentially unsafe upon qualitative examination.

The research was conducted by investigators from Hunan Medical College and Huaihua Central Hospital in Huaihua, China, to address uncertainty regarding whether generative artificial intelligence tools are suitable for patient education in diabetic kidney disease. The study evaluated six widely accessible chatbot platforms: ChatGPT, Gemini, DeepSeek, Doubao, Kimi, and Qwen. Safety was established as the primary outcome measure, alongside structured assessments of accuracy, empathy, information quality, and textual readability.

To test the models, the researchers developed 42 standardized English questions focusing on diabetic kidney disease. Each inquiry was submitted exactly once to each of the six chatbots in independent, single-turn query sessions, producing a dataset of 252 total responses for formal evaluation.

Evaluation of the generated texts was conducted by five clinical raters who were masked to the source models to minimize assessment bias. Readability metrics were evaluated automatically using computational formulas, while inter-rater reliability was measured using Fleiss’ kappa and intraclass correlation coefficients ICC (2,1) and ICC (2,5).

Analysis of safety evaluations demonstrated substantial agreement among the clinical raters, yielding a Fleiss’ kappa of 0.854. Reliability coefficients showed ICC (2,1) values ranging from 0.552 to 0.855 and ICC (2,5) values ranging from 0.860 to 0.967 across the evaluation dimensions. While safe-response rates varied numerically from 83.3 percent to 92.9 percent across the six models, statistical testing using Cochran’s Q test confirmed that the differences in safety between models were not statistically significant with a P-value of 0.726.

In contrast to overall safety, evaluations of response accuracy and clinician-rated empathy showed statistically significant differences across the six chatbots, with both metrics reaching significance at P < 0.001. ChatGPT achieved the highest scores for factual accuracy, whereas Gemini scored highest among the models for clinician-rated empathy. Information quality scores also differed significantly across the platforms, with ChatGPT demonstrating the best overall performance and Doubao recording the lowest scores.

Automated readability assessments revealed that all six evaluated readability indices differed significantly among the chatbots, with all tests showing P < 0.001 and Kendall’s W values ranging from 0.139 to 0.255.

A qualitative review of the 30 responses identified as potentially unsafe, accounting for 11.9 percent of the 252 generated answers, revealed that hazards most frequently stemmed from incomplete medication-related information or insufficiently individualized advice regarding fluid intake. The authors concluded that while publicly accessible large language models can support patient learning, they must not replace professional clinical guidance, underscoring that guideline anchoring, source verification, readability enhancements, and continuous clinical oversight remain essential safeguards.

To evaluate performance differences across the six chatbots, the researchers implemented Friedman tests alongside Cochran’s Q tests, incorporating Kendall’s W to establish effect sizes and applying the Benjamini–Hochberg correction procedure to adjust for multiple testing. While the primary outcome of safety showed no statistically significant differences among the models, secondary evaluations revealed marked disparities, with factual accuracy, clinician-rated empathy, overall information quality, and automated readability metrics all demonstrating statistically significant differences across the evaluated systems.

Although chatbots can generate plausible information, their variable performance across individual quality dimensions and the presence of clinical safety hazards illustrate why their role remains supplementary. The persistence of readability estimates above the sixth-grade reference level further indicates that language complexity remains an obstacle for patient comprehension across all tested models. Consequently, the authors emphasized that clinical oversight, guideline anchoring, source verification, and targeted readability improvements remain indispensable prerequisites before these conversational tools can safely support diabetic kidney disease management.

Subject of Research: Performance and safety evaluation of artificial intelligence chatbots answering patient-facing diabetic kidney disease questions

Article Title: Safety, accuracy, empathy and information quality of six large language models answering patient-facing questions about diabetic kidney disease: a cross-sectional comparative study

Article References: Yu, S., Huang, R., Zhou, W., Duan, N., & Huang, W. (2026). Safety, accuracy, empathy and information quality of six large language models answering patient-facing questions about diabetic kidney disease: a cross-sectional comparative study. Frontiers in Medicine, 13, Article 2007153. https://doi.org/10.3389/fmed.2026.2007153

Image Credits: AI Generated

DOI: 10.3389/fmed.2026.2007153

Keywords: diabetic kidney disease, artificial intelligence, large language models, patient education, chatbots, clinical safety, medical accuracy, health communication, Safety, accuracy, empathy, information

News Source: Jerry Hayes. (November 8, 2026). Six AI Chatbots Show Variable Performance Answering Diabetic Kidney Disease Questions. Scienmag.

Tags: accuracyArtificial Intelligencechatbotsclinical safetyDiabetic kidney diseaseEmpathyhealth communicationinformationLarge Language Modelsmedical accuracyPatient Educationsafety
Share12Tweet7Share2ShareShareShare1

Related Posts

Ruminally Available Methionine Analog Alters Fermentation and Nutrient Digestibility in Laboratory Tests

Ruminally Available Methionine Analog Alters Fermentation and Nutrient Digestibility in Laboratory Tests

November 16, 2026
Multi-Omics Analysis Links Ferroptosis to Diabetes-Induced Erectile Dysfunction

Multi-Omics Analysis Links Ferroptosis to Diabetes-Induced Erectile Dysfunction

November 16, 2026

Higher-Calorie Refeeding Accelerates Paediatric Weight Gain Without Increasing Refeeding Syndrome Risk

November 16, 2026

Sustained Higher Body Mass Index Linked to Elevated Uterine Fibroid Risk

November 16, 2026

POPULAR NEWS

  • Attention Graph Model Improves Dynamic Anomaly Detection Across Six Datasets

    Attention Graph Model Improves Dynamic Anomaly Detection Across Six Datasets

    29 shares
    Share 12 Tweet 7
  • Blockchain-Integrated Machine Learning Enhances Fraud Detection in Supply Chain Networks

    29 shares
    Share 12 Tweet 7
  • Dynamic Cognitive Models Improve External Dementia Transition Predictions But Face Major Limits

    29 shares
    Share 12 Tweet 7
  • Binary Hippopotamus Optimization Achieves High Accuracy Across Sixteen Cancer Datasets

    29 shares
    Share 12 Tweet 7

About

We bring you the latest biotechnology news from best research centers and universities around the world. Check our website.

Follow us

Recent News

ShieldFL Framework Defends Non-IID Federated Learning Against Byzantine Attacks

Surrounding Cereal and Non-Crop Lands Associate With Canola Weevil Numbers

Quantum Inspired Model Directs Metaverse Server Resources in Simulated Cluster Tests

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 87 other subscribers
  • Contact Us

Bioengineer.org © Copyright 2023 All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Homepages
    • Home Page 1
    • Home Page 2
  • News
  • National
  • Business
  • Health
  • Lifestyle
  • Science

Bioengineer.org © Copyright 2023 All Rights Reserved.