LLMs Show Variable Accuracy in Bariatric Surgery Imaging

In a proof-of-concept evaluation assessing the interpretation of postoperative complications after metabolic and bariatric surgery, three multimodal large language models exhibited variable diagnostic performance, with none of the systems achieving full concordance with reference diagnoses across twelve radiologic scenarios. In the comparative evaluation published in BMC Surgery, ChatGPT 5.2 attained the highest overall score by recording 18 out of 24 points, while Gemini-3 Pro and Grok-4.1 each followed with 17 out of 24 points. The investigation examined how the origin of clinical data affects artificial intelligence interpretations, testing each system on published and previously unpublished clinical presentations. The findings indicate that while advanced multimodal systems show potential to support differential diagnosis, their inconsistent accuracy underscores that thorough validation remains necessary before clinical adoption.

Multimodal large language models integrate text and image data to help diagnose conditions, improve consistency, and streamline clinical workflows. These systems have shown promise in medical imaging, demonstrating that they can approach specialist-level performance, support clinical decision-making, and assist in metabolic and bariatric surgery imaging, though clinical applications remain in an early stage. Although large language models frequently perform well on standardized medical examinations, their readiness for real-world medical practice still demands thorough validation before they can be safely adopted at scale.

To evaluate diagnostic reasoning across different information sources, the researchers compared ChatGPT 5.2, Grok-4.1, and Gemini-3 Pro across twelve radiologic scenarios representing postoperative complications following metabolic and bariatric surgery. The twelve scenarios were equally split by data origin, consisting of six unpublished institutional cases and six cases sourced from PubMed. This comparative design allowed the investigators to analyze whether the public availability of case material influenced model interpretations. Each model received standardized prompts along with the corresponding case imaging, and their responses were formally compared against established reference diagnoses.

The study employed top-1 concordance and top-3 inclusion as exploratory performance measures to evaluate the accuracy of the models’ diagnostic proposals. Across the twelve scenarios, ChatGPT 5.2 achieved the highest overall score with 18 out of 24 points. Gemini-3 Pro and Grok-4.1 each followed with 17 out of 24 points. Notably, none of the three models achieved full concordance with the reference diagnoses across all twelve scenarios.

Performance varied substantially depending on the nature of the postoperative complication. All three models demonstrated better performance in identifying structural and anatomical complications than in identifying functional or vascular complications. The models exhibited variable diagnostic accuracy across the clinical spectrum, encountering particular difficulty with less common vascular and bowel-related complications.

The evaluation also demonstrated an observable divergence based on the publication status of the case material. Published cases sourced from PubMed showed better observed diagnostic performance than previously unpublished institutional cases across the models. However, the researchers emphasized that this outcome must be interpreted cautiously. Because publicly available cases may have been represented within the large datasets used during model training, the observed advantage may reflect prior exposure to published case content alongside the recognition of well-documented clinical patterns. Consequently, prior exposure to publicly available content during training cannot be excluded and cannot be considered evidence of superior diagnostic accuracy or generalizability to new clinical settings.

In their conclusion, the authors noted that multimodal large language models demonstrate potential to support the differential diagnosis of postoperative bariatric complications, but their variable performance warrants significant caution. The observed shortcomings in diagnosing less common vascular and bowel-related complications, combined with the unresolved question of training data exposure on publicly available cases, underline the current limits of clinical readiness. Before these automated diagnostic tools can be introduced into clinical implementation, the authors underscored that further evaluation must be conducted using larger datasets composed of demonstrably unseen cases to establish true diagnostic validity.

The report was published as an open-access research article in BMC Surgery on October 9, 2026. According to the publisher, the paper was made available early to accelerate access to peer-reviewed, accepted research, carrying a permanent digital object identifier while remaining subject to further edits before its replacement by the final Version of Record.

Subject of Research: Evaluation of large language models in radiologic interpretation of post-metabolic and bariatric surgery complications

Article Title: Impact of data origin on large language models’ radiologic interpretation: a proof-of-concept study on post-metabolic and bariatric surgery complications

Article References: Kermansaravi, M., Gero, D., ShahabiShahmiri, S., Tolone, S., Jazi, A. H. D., Taherzadeh, M., Mousavimaleki, A., Lainas, P., & Chiappetta, S. (2026). Impact of data origin on large language models’ radiologic interpretation: a proof-of-concept study on post-metabolic and bariatric surgery complications. BMC Surgery. https://doi.org/10.1186/s12893-026-04265-5

Image Credits: AI Generated

DOI: 10.1186/s12893-026-04265-5

Keywords: large language models, metabolic bariatric surgery, radiologic interpretation, medical imaging, AI diagnostics, postoperative complications, ChatGPT 5.2, Grok-4.1, Gemini-3 Pro, proof-of-concept study, BMC Surgery, clinical decision-making, vascular complications, anatomical complications, data origin bias, multimodal AI, diagnostic accuracy, open access research, clinical validation, AI in medicine