ChatGPT-4o Answers on Urinary Incontinence Score High on Accuracy, Study Finds

ChatGPT-4o generally gave accurate, comprehensive and guideline-consistent answers to patient-oriented questions about stress urinary incontinence and pelvic floor muscle exercises, according to a descriptive cross-sectional study published in BMC Women’s Health. Researchers developed 14 patient-oriented questions covering disease-related information, the principles of pelvic floor muscle exercise, exercise implementation and clinical decision-making scenarios, then had experts evaluate the chatbot’s responses.

Two experts independently assessed the two responses for each question together and assigned one score per question for accuracy, comprehensiveness and guideline alignment, producing 14 question-level ratings per domain per evaluator. Initial ratings were recorded separately, and disagreements were resolved through discussion with reference to clinical guidelines and evidence-based literature. Quantitative data were summarized using frequencies, percentages, means with standard deviations, and medians with interquartile ranges, and inter-rater agreement was assessed using percentage agreement.

The results were strikingly uniform on the first two domains. All 14 questions received a final consensus accuracy score of 4, described as “largely correct,” yielding a mean accuracy score of 4.00 with a standard deviation of zero. Likewise, all 14 questions were rated as comprehensive with a score of 3, giving a mean comprehensiveness score of 3.00. Percentage agreement between the evaluators was 85.7 percent for both accuracy and comprehensiveness.

Guideline alignment told a more nuanced story. 64.3 percent of questions were classified as partially compliant and 35.7 percent as fully compliant, corresponding to a mean guideline-alignment score of 2.36 with a standard deviation of 0.50. Inter-rater agreement for this domain was lower, at 78.6 percent. In other words, even when the chatbot’s information was factually sound, it did not always match the specific recommendations of clinical guidelines.

A qualitative content analysis of the responses identified recurring patterns that quantitative scoring alone would miss: scope expansion, unsupported numerical statements, limited contextual specificity and restricted personalization of recommendations.

Stress urinary incontinence is a common health condition that negatively affects women’s quality of life, and pelvic floor muscle exercises are considered the first-line conservative treatment approach.

The study’s authors conclude that ChatGPT-4o demonstrated a high level of accuracy and comprehensiveness in providing information on stress urinary incontinence and pelvic floor muscle training, but that variations in guideline alignment and contextual specificity indicate AI-generated health information should be interpreted with appropriate clinical judgment. They recommend that future evaluations consider not only factual accuracy but also guideline consistency, clinical applicability and the contextual quality of the information provided.

Several limitations shape how the findings should be read. The evaluation covered a single chatbot version, 14 questions in one clinical topic area, and ratings assigned by two experts through consensus discussion, so the scores reflect expert judgment against guidelines rather than patient outcomes or real-world user experiences. The study assessed the content of responses, not whether patients reading them would act correctly or safely, and the qualitative analysis flagged potential patient safety considerations without measuring actual harm. As an evaluation of generated text, it cannot establish how the chatbot would perform on other conditions, in other languages, or after future model updates.

Within those bounds, the message is measured rather than triumphant.

Subject of Research: Expert evaluation of ChatGPT-4o responses to patient questions about stress urinary incontinence and pelvic floor muscle training

Article Title: Evaluation of ChatGPT responses on stress urinary incontinence and pelvic floor muscle training: a guideline-based analysis

Article References: Çakir, P. A., & İnce, B. (2026). Evaluation of ChatGPT responses on stress urinary incontinence and pelvic floor muscle training: a guideline-based analysis. BMC Women's Health. https://doi.org/10.1186/s12905-026-05062-1

Image Credits: AI Generated

DOI: 10.1186/s12905-026-05062-1

Keywords: ChatGPT-4o, stress urinary incontinence, pelvic floor muscle training, guideline alignment, artificial intelligence, women's health, health information, accuracy evaluation, patient education, BMC Women's Health, Evaluation, ChatGPT