Multimodal artificial intelligence models demonstrated low overall agreement with expert orthodontists and poor diagnostic consistency when classifying orthodontic malocclusions from standard two-dimensional digital intraoral images, according to a retrospective diagnostic accuracy study published in BMC Oral Health. Researchers evaluated commercial systems including Claude Opus 4.7, GPT-5.5, and Gemini 3 Flash against an expert reference standard established by two independent orthodontists.
The study cohort comprised 90 patients selected from archived clinical records, deliberately divided into equal groups of 30 across each Angle classification category to ensure balanced representation of dental relationships. For each patient, five standardized two-dimensional image views exported from iTero intraoral digital scans were presented to the three artificial intelligence systems using a single zero-shot prompt. To establish the ground-truth reference standard, two independent orthodontists evaluated the full interactive three-dimensional digital scan records and conducted precise metric measurements, arriving at agreed clinical classifications across all evaluated parameters.
Evaluated diagnostic parameters included sagittal dental relationship classification, vertical relationship, crowding severity, and arch form. Across the models evaluated, overall diagnostic accuracy demonstrated substantial variability but remained low overall, ranging from a low of 29.2 percent for Gemini 3 Flash when assessing dental arch form to a maximum of 64.0 percent for GPT-5.5 on the same parameter. Macro-average recall across all tested categories similarly showed limited diagnostic performance, spanning from 38.7 percent for Gemini 3 Flash assessing arch form to 60.9 percent for GPT-5.5 classifying vertical relationships.
Statistical evaluation of classification agreement further highlighted significant discrepancies between model predictions and expert reference findings. Unweighted Cohen’s kappa values ranged from 0.044 for Gemini 3 Flash on arch form classification to 0.299 for the same model when assessing crowding severity. Category-specific performance metrics revealed pronounced sensitivity disparities across different types of malocclusion. While the models achieved sensitivity rates between 60.0 percent and 80.0 percent for Angle Class I cases, sensitivity dropped sharply to between 10.0 percent and 13.3 percent when detecting Class III malocclusions.
When classifying all cases from the same five images, the orthodontists agreed with the reference standard in 96.6 to 100 percent of cases.
Diagnostic reproducibility was evaluated by resubmitting a subset of 20 patient cases to the models after an interval of four weeks under identical testing conditions. Test-retest agreement metrics revealed severe inconsistency over time, with Cohen’s kappa values ranging from negative 0.238 for Gemini 3 Flash on sagittal classification to 0.433 for GPT-5.5 on vertical relationships, with most 95 percent confidence intervals spanning zero. In this retest subsample, sagittal accuracy dropped from an initial range of 55.0 to 60.0 percent down to between 5.0 and 35.0 percent, whereas both human orthodontists reproduced at least 19 of 20 classifications across every evaluated parameter.
The authors concluded that under the experimental conditions evaluated, the three multimodal models exhibited low agreement with expert orthodontic reference standards, marked underperformance on Class III malocclusion identification, and unacceptable diagnostic variability over time. Furthermore, aggregate accuracy figures failed to reflect critical category-specific diagnostic weaknesses, demonstrating that diagnostic reproducibility must be assessed independently from accuracy metrics. Consequently, the researchers determined that commercial multimodal language models remain unsuitable for autonomous or standalone orthodontic diagnostic tasks.
The authors noted that while multimodal artificial intelligence systems built on large language models are consulted by patients and clinicians, their performance on visual orthodontic classification is not well characterized.
Furthermore, the researchers applied ninety-five percent confidence intervals across all evaluated parameters, including overall accuracy, macro-average recall, and unweighted Cohen’s kappa values.
Subject of Research: Diagnostic accuracy and reproducibility of multimodal large language models in orthodontic classification
Article Title: Large language models for orthodontic classification using intraoral digital models: a diagnostic accuracy and reproducibility study
Article References: Altundal, G., Kürkçü, İ., Altın, M., & Biren, S. (2026). Large language models for orthodontic classification using intraoral digital models: a diagnostic accuracy and reproducibility study. BMC Oral Health. https://doi.org/10.1186/s12903-026-10465-5
Image Credits: AI Generated
DOI: 10.1186/s12903-026-10465-5
Keywords: orthodontics, large language models, artificial intelligence, diagnostic accuracy, reproducibility, Angle classification, multimodal AI, digital dentistry, intraoral scans, iTero, Claude Opus, GPT-5.5, Gemini Flash, Cohen's kappa, Class III malocclusion, dental alignment, clinical diagnostics, machine learning in dentistry
News Source: Ophelia Keating. (November 19, 2026). Multimodal Large Language Models Show Low Accuracy in Orthodontic Classification Study. Scienmag.



