Logistic Regression Matches Machine Learning in Thyroid Nodule Cancer Risk Stratification

In a single-centre retrospective cohort study assessing thyroid nodule malignancy risk, multivariable logistic regression matched or outperformed several machine learning models under strict temporal validation. Conducted by researchers Juan Jesús Fernández Alba, Florentino Carral San Laureano, Ana Isabel Jiménez Millán, and Carmen González Macías, the study tested whether machine learning could improve cancer risk stratification using routinely collected clinical, biochemical, and ultrasound predictors. The investigators found that none of the evaluated machine learning algorithms demonstrated improved performance over a well-specified multivariable logistic regression model.

Ultrasound risk-stratification systems, including ACR TI-RADS, EU-TIRADS, and ATA 2015, currently guide fine-needle aspiration decisions for thyroid nodules, but these systems do not integrate clinical or biochemical information. Multivariable logistic regression models incorporating these clinical and biochemical variables provide individualised malignancy probability estimates to aid clinical assessment. While machine learning algorithms have been proposed as potential improvements over conventional regression, rigorous comparative evidence under strict temporal validation has remained scarce.

To address this knowledge gap, the authors conducted a single-centre retrospective cohort study examining 861 thyroidectomies performed between 2010 and 2023. The training cohort consisted of 432 patients treated before 2019, including 94 confirmed malignant cases. The independent temporal test cohort included 429 patients treated from 2019 onwards, containing 92 malignant cases.

The evaluated logistic regression model used the exact specification of a previously published and externally validated model, with its coefficients re-estimated directly in the training cohort. The authors compared this model head-to-head with Random Forest, XGBoost, Bagged CART, and a multilayer perceptron neural network trained on the exact same predictors. Performance was comprehensively assessed by discrimination using ROC-AUC and precision-recall AUC, calibration using the Brier score along with calibration slope and intercept, and clinical utility using decision curve analysis, reporting operating characteristics at the pre-specified 9.55 percent probability threshold.

Under temporal validation, logistic regression achieved an area under the receiver operating characteristic curve of 0.838 (95% CI 0.789–0.884), closely matching the 0.84 reported in the original model’s published external validation. Performance among the tree-based machine learning models was statistically indistinguishable from logistic regression: Random Forest reached an AUC of 0.843 (p = 0.76), while XGBoost attained an AUC of 0.844 (p = 0.69). Bagged CART produced a non-significantly inferior AUC of 0.821 (p = 0.40).

In additional performance metrics, logistic regression showed the highest precision-recall AUC at 0.672 and demonstrated good calibration without requiring post-hoc adjustment, yielding a calibration slope of 1.12 with a 95 percent confidence interval of 0.91 to 1.43. When compared against standard clinical practice, logistic regression, Random Forest, XGBoost, and Bagged CART all discriminated significantly better than the ATA 2015 ultrasound classification, which achieved an AUC of 0.750 with DeLong test p-values of 0.002 or less. By contrast, the multilayer perceptron did not differ significantly from the ATA 2015 classification, recording a p-value of 0.45.

In exploratory subgroup analyses by Bethesda category, nodule size, and age, no evidence of a consistent machine learning advantage was observed, though the researchers noted that several subgroups were underpowered. Explainability assessments demonstrated that SHAP importance rankings from Random Forest and XGBoost showed broadly similar global orderings to the logistic regression coefficients, yielding correlation values of 0.71 and 0.77. The researchers concluded that in this single-centre surgical cohort with a moderate sample size and routinely collected structured predictors, none of the evaluated machine learning algorithms demonstrated improved performance over a well-specified logistic regression model.

Subject of Research: Comparative performance of logistic regression and machine learning for thyroid nodule risk stratification

Article Title: Logistic regression withstands comparison with machine learning for thyroid nodule malignancy risk stratification under strict temporal validation

Article References: Fernández Alba, J. J., Carral San Laureano, F., Jiménez Millán, A. I., & González Macías, C. (2026). Logistic regression withstands comparison with machine learning for thyroid nodule malignancy risk stratification under strict temporal validation. PLOS Digital Health, 5(11), e0001828. https://doi.org/10.1371/journal.pdig.0001828

Image Credits: AI Generated

DOI: 10.1371/journal.pdig.0001828

Keywords: thyroid nodules, logistic regression, machine learning, temporal validation, risk stratification, malignancy prediction, random forest, xgboost, bagged cart, multilayer perceptron, thyroidectomy, fine-needle aspiration, ATA 2015, ROC-AUC, decision curve analysis, SHAP values, Bethesda category