A comparative, web-based cross-sectional study published in Frontiers in Medicine evaluated the performance of five publicly accessible large language models answering patient-oriented questions regarding irritable bowel syndrome.
To assemble a representative query set, the investigators retrieved search data from Google Trends across the preceding five years using the English Medical Subject Headings term for irritable bowel syndrome under a worldwide geographic filter. Following a screening protocol, the researchers finalized twelve patient-oriented questions addressing critical aspects of the condition. These standardized prompts were submitted in English to five distinct computational platforms: ChatGPT, Gemini, Microsoft Copilot, Perplexity, and DeepSeek. Each model operated under a standardized single-turn, zero-shot testing protocol, generating exactly one response per query and yielding a total sample of sixty evaluated outputs.
Evaluation of the collected material was conducted independently by two gastroenterologists using established clinical information assessment instruments. The reviewers scored the sixty responses utilizing the DISCERN instrument for treatment choices, the Ensuring Quality Information for Patients tool, the Global Quality Score, and benchmark transparency criteria established by the Journal of the American Medical Association. Information-quality scores across configurations were formally compared using question-blocked Friedman tests, accompanied by post hoc Bonferroni-adjusted paired Wilcoxon signed-rank tests when omnibus analyses revealed significant variation. In parallel, textual readability was evaluated across six recognized formulas: the Automated Readability Index, the Gunning Fog Index, the Flesch-Kincaid Grade Level, the Coleman-Liau Index, the Simple Measure of Gobbledygook, and the Flesch Reading Ease Score, adopting a sixth-grade reading level as the predefined reference standard.
Statistical testing identified significant omnibus differences among the evaluated models across DISCERN, Ensuring Quality Information for Patients, and JAMA transparency scores, with permutation P values falling below 0.001 for each metric. Conversely, the Global Quality Score demonstrated no statistically significant difference across platforms, yielding a permutation P value of 0.600 and a uniform median rating of 4. Pairwise analyses indicated that Microsoft Copilot and Perplexity achieved the highest median DISCERN scores at 48.00. Copilot attained the highest median Ensuring Quality Information for Patients score at 80.00, significantly outperforming ChatGPT, Gemini, and DeepSeek in Bonferroni-adjusted paired comparisons.
Despite relatively superior performance on specific quality metrics by selected tools, source transparency remained universally constrained across all evaluated platforms. Every model configuration registered median JAMA benchmark scores of only 0 to 1, indicating limited fulfillment of the assessed transparency-related criteria. Although Copilot and Perplexity achieved relatively higher JAMA scores, transparency remained limited.
Readability assessments revealed that no model met the predefined sixth-grade readability standard. Median scores across the Automated Readability Index, Gunning Fog Index, Flesch-Kincaid Grade Level, Coleman-Liau Index, and Simple Measure of Gobbledygook uniformly exceeded grade 6, while all median Flesch Reading Ease scores fell below 80. Gemini recorded the highest median difficulty scores across all five grade-level indices. Copilot recorded the lowest medians for the Automated Readability Index, Gunning Fog Index, Flesch-Kincaid Grade Level, and Simple Measure of Gobbledygook, whereas DeepSeek registered the lowest median Coleman-Liau score and highest median Flesch Reading Ease score.
The study concluded that while all five evaluated large language models generated irritable bowel syndrome materials possessing baseline educational value, they diverged markedly in treatment quality, content completeness, and source attribution. Because reading levels remained consistently elevated above standard patient literacy recommendations, the authors emphasize that current artificial intelligence platforms should function solely as professionally supervised educational adjuncts rather than standalone resources for medical diagnosis, treatment planning, or clinical decision-making.
Subject of Research: Evaluation of artificial intelligence models answering irritable bowel syndrome questions
Article Title: Reliability and readability of five large language models in answering patient-oriented questions about irritable bowel syndrome: a comparative study
Article References: Liang, L., Zhang, L., Wang, H., Peng, X., Zhang, X., Liu, S., Wang, T., Qu, P., Han, J., & Li, Z. (2026). Reliability and readability of five large language models in answering patient-oriented questions about irritable bowel syndrome: a comparative study. Frontiers in Medicine, 13, Article 1964767. https://doi.org/10.3389/fmed.2026.1964767
Image Credits: AI Generated
DOI: 10.3389/fmed.2026.1964767
Keywords: irritable bowel syndrome, artificial intelligence, large language models, health literacy, patient education, readability, information quality, gastroenterology, Reliability, five, large, language
News Source: Ophelia Keating. (November 9, 2026). Five AI Models Evaluated for Irritable Bowel Syndrome Patient Information. Scienmag.



