AI Reads Ovarian Ultrasounds Well, But Bias and Thin Validation Hold It Back

Artificial intelligence is being applied to ultrasound imaging of polycystic ovary syndrome with strikingly high reported accuracy, yet a new systematic review warns that the field’s enthusiasm may be running ahead of its evidence. The analysis, published in the Journal of Ovarian Research, examined 65 studies published between 2017 and 2026 and found that while individual AI models frequently achieved strong diagnostic performance, direct comparison between them was nearly impossible because of wildly inconsistent datasets, imaging protocols, and validation strategies.

Polyendocrine metabolic ovarian syndrome, formerly called polycystic ovary syndrome, is a heterogeneous endocrine and metabolic disorder with both reproductive and metabolic manifestations. Ultrasound remains the primary imaging modality for assessing polycystic ovarian morphology within the historical diagnostic framework, but its interpretation is hampered by operator dependence, interobserver variability, image quality problems, and subjective assessment. These are precisely the weaknesses that machine learning and deep learning promise to address through automated image analysis, follicle segmentation, feature extraction, and diagnostic classification.

The review’s authors, Piyush Jagdish Balgote and Jayanthi Sivaraman, conducted a structured search of major biomedical and engineering databases to assemble their evidence base. They focused specifically on AI-assisted ultrasound imaging, including multimodal studies only when ultrasound formed an integral component of the diagnostic framework.

Methodological scrutiny was central to the review. The researchers assessed methodological quality and risk of bias in 53 eligible prediction-model studies using the Prediction model Risk of Bias Assessment Tool, known as PROBAST. They identified substantial risk-of-bias concerns, particularly in the analysis domain, and visualised the results using the ROBVIS tool. This formal bias assessment identified substantial risk-of-bias concerns, particularly in the analysis domain.

The core problem is heterogeneity. Studies varied in the datasets they used, the imaging protocols they followed, how they defined diagnostic outcomes, how they preprocessed images, how they developed their models, and how they validated results. Reported diagnostic performance was generally high across individual studies, but direct comparison was limited by heterogeneity in datasets, imaging protocols, outcome definitions, preprocessing, model development, and validation strategies.

Several structural barriers compound the problem. The review found frequent reliance on retrospective, single-centre datasets, limited external validation, and inconsistent reporting. The review found frequent reliance on retrospective, single-centre datasets, limited external validation, and inconsistent reporting, which constrain confidence in the generalisability and clinical applicability of existing AI models.

Beyond statistical concerns, the authors identify practical obstacles to clinical translation. These include interpretability, meaning clinicians need to understand why a model reaches its conclusions; data governance, covering how patient images are collected, stored, and shared; regulatory requirements that medical AI systems must satisfy before deployment; and workflow integration, the challenge of fitting automated analysis into real clinical routines without disrupting care.

Future work should prioritise standardised reporting, multicentre datasets representing diverse populations, prospective external validation, transparent model development, and regulatory-compliant evaluation. Only through such measures, they argue, can the reliability and clinical applicability of AI-assisted ultrasound systems for PCOS diagnosis be established.

Subject of Research: Artificial intelligence applied to ultrasound imaging for diagnosing polycystic ovary syndrome, with assessment of methodological bias and barriers to clinical translation

Article Title: Artificial intelligence in ultrasound imaging for polycystic ovary syndrome (PCOS): methodological advances, bias, and clinical translation

Article References: Balgote, P. J., & Sivaraman, J. (2026). Artificial intelligence in ultrasound imaging for polycystic ovary syndrome (PCOS): methodological advances, bias, and clinical translation. Journal of Ovarian Research. https://doi.org/10.1186/s13048-026-02247-y

Image Credits: AI Generated

DOI: 10.1186/s13048-026-02247-y

Keywords: artificial intelligence, ultrasound imaging, polycystic ovary syndrome, machine learning, deep learning, risk of bias, PROBAST, external validation, diagnostic classification, clinical translation, follicle segmentation, polyendocrine metabolic ovarian syndrome