Mixed-effects radiomics model hits AUC 0.91 for lung cancer prediction across seven UK hospitals

A radiomics model built to predict whether indeterminate lung nodules are cancerous has achieved an area under the curve of 0.91 in cross-validation across data from seven UK hospitals, according to results from the Lung Imaging Biobank for Radiomics and AI (LIBRA) study published in European Radiology Experimental. The retrospective study, which analysed 2,071 computed tomography scans from 1,026 patients, found that the way a model handles differences between hospitals matters enormously: a mixed-effects approach that treated study centre as a random effect outperformed both ComBat batch correction and fixed-effect adjustments, with between-centre discrimination ranging from 0.75 to 0.91 depending on the method used.

The clinical problem the researchers set out to address is a growing one. Diagnostic CT use is increasing globally, and incidental pulmonary nodules are detected in an estimated 1.6 million people annually in the USA alone. The vast majority of these nodules are benign, so identifying the small proportion—roughly 3 to 5 per cent—that represent early-stage lung cancers is genuinely difficult. Existing guideline tools such as the Brock risk score help stratify nodules, but reliance on serial follow-up scans generates uncertainty for patients and consumes considerable healthcare resources, a pressure sharpened by workforce concerns following the COVID-19 pandemic.

Deep learning models have shown promise in this space. A convolutional neural network developed by Google researchers using the USA National Lung Cancer Screening Trial data achieved an AUC of 0.94, and Optellum’s Lung Cancer Prediction CNN, built on the same trial data, was validated in three UK centres with an AUC of 0.90. But these approaches require very large training datasets, and their internal logic can be difficult to explain. Researchers have also raised concerns that the ‘black box’ nature of such algorithms may limit patient acceptability, and that commercially licensed proprietary code could restrict scientific transparency and reproducibility.

Hand-crafted radiomics offers an alternative philosophy. Rather than learning features automatically, radiomics extracts large numbers of quantitative measurements—texture, intensity and shape descriptors—from manually segmented tumour regions using open-source libraries such as PyRadiomics, making the pipeline more transparent. The Imaging Biomarker Standardisation Initiative has worked to calibrate feature extraction programmes across institutions to improve reproducibility. Yet the LIBRA team noted that few radiomics models have been developed using heterogeneous, clinically acquired CT images from multiple centres, and the impact of study centre on performance is often overlooked. How best to account for centre-specific effects on radiomics features has remained unclear.

The LIBRA study recruited patients aged over 18 with indeterminate pulmonary nodules of 5 to 30 millimetres on baseline CT between 1 July 2020 and 30 September 2022 at seven UK hospitals. Benign nodules had to meet British Thoracic Society criteria: shrinking, stable for one year on volumetry, or stable for two years by transaxial diameter, with subsolid nodules requiring four years of stability. All malignant nodules were biopsy-proven. Scans thicker than 2.5 millimetres and nodules with unknown ground truth were excluded. Up to three scans per patient were included, and scans were grouped by fold identifier to prevent data leakage when splitting. Of the cohort, 491 patients (48 per cent) were male, 456 (65 per cent of those recorded) had a positive smoking history, and 560 (55 per cent) had benign nodules.

Image processing was deliberately standardised. DICOM images were anonymised at each centre and uploaded to a study XNAT server, converted to NIfTI format, and manually segmented by a consultant clinical oncologist with nine years’ experience using ITK-Snap. Images and masks were resampled to 1 cubic millimetre, and 107 features were extracted with PyRadiomics using a binwidth of 25 Hounsfield units. To guard against unstable features, the RIDER lung reference dataset was used to assess test–retest reproducibility and inter-reader segmentation variability; 42 features with intraclass correlation coefficients of at least 0.8 were retained. After removing highly correlated features, principal component analysis reduced dimensionality to 20 components explaining 100 per cent of the data variation, with the first five components explaining 86 per cent of the variance.

Three modelling strategies were compared, each including a fixed effect for nodule density and adjustment for contrast status. The first applied ComBat batch correction for study centre; the second used fixed effects for centre; the third used random effects for centre in a mixed-effects model. The ComBat model performed worst, with an AUC of 0.75 (95 per cent confidence interval 0.72–0.78). The mixed-effects model achieved an AUC of 0.91 (95 per cent CI 0.90–0.93) in ten-fold cross-validation, significantly better than the fixed-effects model’s 0.88 (95 per cent CI 0.86–0.90), with p values of less than 0.0001 and 0.003 respectively. These advantages persisted when the Royal Marsden Hospital—which contributed only malignant nodules from an older cohort—was excluded from all models. Because RMH’s malignant-only composition made fixed-effect estimates unstable, those patients were removed from the fixed-effects model.

Benchmarking confirmed that texture information, not just size, drove the model’s performance. The radiomics model significantly outperformed a volume-only model (AUC 0.87, 95 per cent CI 0.86–0.87) and a scrambled-voxel ‘null’ model (AUC 0.75, 95 per cent CI 0.74–0.75). A clinical variable-only model built from age, smoking status, nodule size, upper lobe location, spiculation, intravenous contrast, non-solid density, male sex and multiple nodules achieved an AUC of 0.87 (95 per cent CI 0.86–0.89), significantly lower than the radiomics model. Adding the 20 radiomics components to clinical variables improved the AUC to 0.92 (95 per cent CI 0.91–0.93) and improved model fit, with the Akaike information criterion falling from 1,684 to 1,336. The final model was well calibrated, with an intercept of −0.036 and a slope of 0.89, and an informative Brier score of 0.11 against a non-informative threshold of 0.23.

Decision-curve analysis showed net benefit over volume alone, the Brock score alone, and both ‘investigate all’ and ‘investigate none’ strategies. At 10 per cent and 20 per cent risk thresholds, the model’s net benefit was 0.33 and 0.30 respectively, compared with 0.30 and 0.27 for the Brock score. However, the authors caution that classification at these thresholds was suboptimal: no malignant nodules appeared in the 10 and 20 per cent risk groups, yielding 63 per cent accuracy with zero sensitivity. Using a probability threshold of 0.3186 that maximised the Youden index, example metrics were 0.82 accuracy (95 per cent CI 0.81–0.84), 0.86 sensitivity, 0.80 specificity, 0.72 positive predictive value and 0.91 negative predictive value.

The team is candid about limitations. There was no predefined external test set, which could increase the risk of overfitting, and the recruitment target of 1,000 patients was based on expert consensus rather than a formal power calculation because of the exploratory nature of the work. Performance varied between centres, possibly related to nodule size: the two worst-performing centres had nodules averaging 20.5 millimetres in diameter versus 13.7 millimetres elsewhere. The model currently depends on manual segmentation and a multi-step pipeline, and how it might assist human readers in nodule classification has not been investigated. The authors stress that the model, which maintained good performance across centres in internal-external validation, requires further validation in an independent dataset before it could assist clinical decision-making, and that reader studies with radiologists and integration into a single streamlined clinical workflow are areas for future development.

Subject of Research: Radiomics machine learning models for predicting malignancy in indeterminate pulmonary nodules on multicentre chest CT

Article Title: Radiomics machine learning models for lung cancer early diagnosis in heterogenous multicentre chest CT data: LIBRA study results

Article References: Hunter, B., Ratnakumar, P., Tong, D., Senthivel, N., Bhamani, A., Coe, A., Bloch, S., Kemp, S. V., Doran, S., Navani, N., Nair, A., Ratoff, J., Wells, C. E., Garner, J., Nicolson, A., Bunce, C., Blackledge, M., Devaraj, A., Aboagye, E. O., & Lee, R. W. (2026). Radiomics machine learning models for lung cancer early diagnosis in heterogenous multicentre chest CT data: LIBRA study results. European Radiology Experimental, 10(1), Article 149. https://doi.org/10.1186/s41747-026-00791-2

Image Credits: AI Generated

DOI: 10.1186/s41747-026-00791-2

Keywords: radiomics, lung cancer, pulmonary nodules, chest CT, machine learning, mixed-effects model, LIBRA study, multicentre validation, Brock score, PyRadiomics, cross-validation, clinical decision support