Study Compares AI Models on Neck Pain Safety

A new study published in BMC Musculoskeletal Disorders compared the performance of two large language models, ChatGPT-5.2 and Gemini 3 Pro, in the context of neck pain and interventional pain management. The research, authored by Ebru Alanbay Yagci and Tuğçe Yavuz Mollavelioğlu, utilized a multidimensional framework to evaluate clinical performance, safety, readability, expert assessment, and short-term temporal consistency. Large language models are increasingly used by patients and clinicians to obtain health-related information, yet their reliability in neck pain and interventional pain management remains uncertain.

The study employed a standardized, investigator-developed set of fifty neck pain-related questions. These questions covered patient-oriented inquiries, interventional pain management scenarios, safety and red-flag situations, and clinical vignettes. To test for temporal consistency, the same questions were submitted to both models at two distinct time points: Week 0 and Week 2. For each model, question, and time-point combination, one response was obtained. Responses were anonymized, and evaluators were blinded to model identity.

Evaluation metrics included accuracy, completeness, usefulness, and guideline adherence, which were assessed using predefined five-point ordinal Likert-type scales. Binary outcomes were used to evaluate specific safety risks, including red-flag omissions, hallucinated facts, and overconfident inappropriate recommendations. Readability was specifically assessed for patient-oriented questions using the Flesch–Kincaid Grade Level metric. Additionally, two independent evaluators assigned reviewer comment scores to the responses. The analyses were exploratory in nature, with no formal adjustment for multiple comparisons, and p-values less than 0.05 were considered nominally statistically significant.

A total of two hundred responses were evaluated across the two models and two time points. At Week 0, Gemini 3 Pro demonstrated higher scores for completeness and usefulness compared to ChatGPT-5.2. However, no nominally significant differences were observed between the models regarding accuracy or guideline adherence at this initial time point. Accuracy scores showed no nominally significant difference between the models at either time point.

Safety metrics revealed notable differences in specific areas. Red-flag omissions were more frequent in responses from ChatGPT-5.2 than Gemini 3 Pro at Week 0, with rates of 24% versus 8%, respectively. This difference was nominally statistically significant with a p-value of 0.008. At Week 2, no nominally significant between-model difference was observed for red-flag omissions. Furthermore, no nominally significant differences were found between the models regarding hallucinated facts or overconfident inappropriate recommendations at either time point. In the specific subgroup of interventional pain management questions, no nominally significant between-model differences were observed for any of the evaluated outcomes.

Readability analysis indicated that ChatGPT-5.2 produced text with higher Flesch–Kincaid Grade Level scores at both time points. This suggests that the textual complexity of ChatGPT-5.2’s responses was greater than that of Gemini 3 Pro. The inter-rater reliability for the reviewer comment scores ranged from good to excellent, indicating a high level of agreement between the two independent evaluators.

The authors noted that several within-model differences between time points were nominally significant but were interpreted as short-term variation rather than evidence of true longitudinal change. The study concludes that while both ChatGPT-5.2 and Gemini 3 Pro generally received high scores within the predefined evaluation framework, the between-model differences varied by outcome and assessment time point. Between-model differences did not consistently favor either model. The findings support the necessity of multidimensional, time-specific evaluation of large language models in medical contexts.

The study emphasizes that large language models should be regarded as informational or decision-support tools rather than substitutes for professional clinical reasoning or individualized clinical decision-making. This applies particularly in safety-critical or interventional contexts.

Subject of Research: Comparison of ChatGPT-5.2 and Gemini 3 Pro performance in neck pain and interventional pain management

Article Title: ChatGPT-5.2 versus Gemini 3 Pro in neck pain and interventional pain management: a multidimensional comparison of performance, safety, readability, and short-term temporal consistency

Article References: Alanbay Yagci, E., & Yavuz Mollavelioğlu, T. (2026). ChatGPT-5.2 versus Gemini 3 Pro in neck pain and interventional pain management: a multidimensional comparison of performance, safety, readability, and short-term temporal consistency. BMC Musculoskeletal Disorders. https://doi.org/10.1186/s12891-026-10552-2

Image Credits: AI Generated

DOI: 10.1186/s12891-026-10552-2

Keywords: ChatGPT-5.2, Gemini 3 Pro, Neck Pain, Interventional Pain Management, Large Language Models, Medical AI, Safety Assessment, Readability, Temporal Consistency, BMC Musculoskeletal Disorders, ChatGPT-5, versus