Large Language Models Surpass Passing Threshold on OB-GYN Certification Exams

A recent study published in the Archives of Gynecology and Obstetrics evaluated the performance of six widely used large language models on specialty certification examinations in obstetrics and gynecology. The research team, led by Anna Owczarek, Franciszek Ługowski, and Paweł Jan Stanirowski, aimed to assess the utility of these artificial intelligence systems as educational tools and adjunctive knowledge-assistance systems. The study found that all tested models surpassed the 60% passing threshold required for human specialists, although significant performance disparities were observed among them.

The investigation focused on a pragmatic sample of current large language models, including ChatGPT 5.1, Gemini Flash 2.5, Gemini Pro 3.0, Claude Sonnet 4.5, Microsoft Copilot, and Llama 4 Maverick. The researchers noted that while large language models are becoming integral to medical fields, comprehensive evaluations of their expertise in specific domains such as obstetrics and gynecology had remained limited prior to this work.

To conduct the assessment, the authors submitted 355 single-best-answer questions to the models. These questions were drawn from three recent sessions of the Polish Specialty Certificate Examination in Obstetrics and Gynecology. The questions were translated into English before being presented to the artificial intelligence systems. Model performance was evaluated based on overall accuracy and the difficulty index of the questions.

Gemini Pro 3.0 achieved superior results, consistently exceeding 90% accuracy and maintaining stability regardless of question complexity.

In contrast, descriptive session-stratified analyses indicated lower accuracy for Gemini Flash 2.5, Microsoft Copilot, and Llama 4 Maverick on difficult questions.

The study suggests that while large language models hold promise as educational and knowledge-assistance tools, their performance variability necessitates rigorous prospective validation and expert oversight before any clinical deployment.

Subject of Research: Large language models and specialty certification examinations in obstetrics and gynecology: a comparative performance analysis

Article Title: Large language models and specialty certification examinations in obstetrics and gynecology: a comparative performance analysis

Article References: Owczarek, A., Ługowski, F., & Stanirowski, P. J. (2026). Large language models and specialty certification examinations in obstetrics and gynecology: a comparative performance analysis. Archives of Gynecology and Obstetrics. https://doi.org/10.1007/s00404-026-08572-3

Image Credits: AI Generated

DOI: 10.1007/s00404-026-08572-3

Keywords: Large, language, models, specialty, certification, examinations, obstetrics, gynecology, comparative, performance, analysis, scientific research