AI Reads CT Scans to Predict Esophageal Cancer Response, With Mixed Results

For elderly patients with esophageal cancer, the weeks between starting definitive radiotherapy and learning whether the tumor is actually shrinking can feel like an eternity. A new proof-of-concept study from China suggests that artificial intelligence might one day shorten that wait, but the research also delivers a sobering lesson about how easily machine learning predictions can crumble under rigorous statistical scrutiny. The study, published in BMC Medical Imaging by a team at the First Affiliated Hospital of the University of Science and Technology of China and Anhui Provincial Cancer Hospital, set out to determine whether subtle patterns invisible to the human eye on routine CT scans could forecast which patients would respond early to treatment.

The technique at the heart of the study is called radiomics, a branch of medical imaging analysis that converts ordinary pictures into vast troves of quantitative data. Where a radiologist sees a tumor’s size, shape, and density, a radiomics pipeline extracts hundreds of mathematical descriptors: measures of texture homogeneity, gray-level co-occurrence patterns, edge sharpness, and spatial complexity. The premise is seductive. Tumors that are biologically primed to die under radiation may carry telltale signatures in their pixel-level architecture, signatures that could be captured before a single treatment session begins. If such signatures could be reliably decoded, oncologists could escalate therapy for predicted non-responders, spare frail elderly patients the burden of ineffective treatment, and personalize care in ways currently impossible.

The research team retrospectively reviewed 189 patients with esophageal cancer who had undergone definitive radiotherapy at their single center. All patients had baseline contrast-enhanced CT scans from which radiomic features were extracted. The cohort divided into 126 responders and 63 non-responders based on early radiological assessment of treatment effect. Following international reporting standards known as TRIPOD, the investigators pre-specified their primary analysis: a LASSO-penalized logistic regression model, a statistical technique that shrinks the influence of weak predictors toward zero and selects only the most informative features. Crucially, they capped the number of predictors at five, a constraint dictated by the number of non-responders in the cohort, since reliable model fitting generally demands at least ten outcome events per predictor variable.

What followed is a masterclass in methodological honesty, and it is here that the study earns its place in the ongoing debate about artificial intelligence in medicine. When the team evaluated their five-feature radiomic model using fully nested cross-validation, a procedure in which every step of the analysis, from initial filtering through feature selection to final model fitting, is re-executed from scratch within each testing fold, the model’s discrimination collapsed to an area under the curve of just 0.554, with a standard deviation of 0.108. An AUC of 0.5 represents performance no better than a coin flip. In other words, when the researchers refused to let any information from the test data leak into the training process, the radiomic model’s apparent predictive power essentially evaporated.

The contrast with less rigorous evaluation strategies is striking and instructive. When the univariable filtering step was performed just once on the entire training cohort rather than repeated within each fold, the model’s estimated AUC rose to 0.637, with an optimism correction of 0.148 revealing how much the apparent performance of 0.785 had been inflated. On the small independent test set of 38 patients, the model posted an AUC of 0.716, a figure that looks respectable on paper but rests on so few cases that the confidence interval around it is wide enough to drive a truck through. Calibration analysis, which assesses whether predicted probabilities match observed outcomes, yielded a Brier score of 0.216, with a calibration slope of 1.502 suggesting predictions that were somewhat too timid in their extremes.

The team also explored, explicitly labeled as exploratory, a family of support vector machine models trained on three different feature sets: clinical factors alone, radiomic features alone, and a combination of both. Support vector machines are algorithms that seek the optimal boundary separating two classes in a high-dimensional space, and they remain popular in radiomics research for their ability to handle complex, nonlinear relationships. On the test set, the clinical-only model achieved an AUC of 0.603, the radiomics-only model 0.732, and the multimodal model 0.745. At first glance, the multimodal approach appears to win. But statistical testing told a different story. Using bootstrap paired comparisons with 5,000 resamples, the researchers found that the multimodal model’s advantage over the clinical-only model, a difference of 0.142 in AUC, carried a 95 percent confidence interval stretching from minus 0.077 to plus 0.364, with a p-value of 0.212. Against the radiomics-only model, the increment was a negligible 0.012, clearly non-significant.

This failure to demonstrate incremental value cuts to the core of the multimodal radiomics hypothesis. The entire rationale for combining imaging data with clinical variables is that each source contributes unique information. If the combined model performs no better than either component alone, the added complexity, computational cost, and validation burden of multimodal pipelines may not be justified. The study also examined whether age modified performance, a relevant question given the elderly cohort, and found multimodal discrimination was broadly similar across age strata, with AUCs of 0.739 versus 0.786 in the compared groups. Age, in other words, did not rescue the model’s performance or meaningfully stratify it.

Why did the radiomic signals prove so fragile? Several explanations deserve consideration. First, the sample size is modest by machine learning standards, and the 63 non-responders imposed a hard ceiling on model complexity. Second, radiomic features are notoriously sensitive to variations in CT acquisition parameters, reconstruction kernels, and contrast timing, all of which can vary even within a single center over time. Third, and perhaps most fundamentally, the feature selection process itself is a source of instability. When hundreds of candidate features are filtered and winnowed on slightly different subsets of data, different features can emerge as winners each time, a phenomenon the researchers probed with a 200-resample feature-stability analysis. The gap between the partially nested estimate of 0.637 and the fully nested estimate of 0.554 quantifies precisely how much of the model’s apparent skill was an artifact of information leakage rather than genuine biology.

The authors are refreshingly candid about these limitations, concluding that their pre-specified model achieves limited and unstable discrimination and that no evidence supports incremental value for combining clinical and radiomic data. They call for prospective multicenter validation before any clinical application, a bar that most published radiomics studies never clear. The study was conducted without external funding, supported entirely by the routine clinical work of the Department of Radiation Oncology, and approved by the ethics committee of Anhui Provincial Cancer Hospital with informed consent waived for the retrospective design.

In an era when headlines routinely proclaim that artificial intelligence can outperform physicians, this study offers a quieter and more valuable contribution: a demonstration of what rigorous, pre-specified, leakage-free evaluation actually looks like, and a reminder that the gap between apparent and real predictive performance can swallow an entire research program. For elderly patients with esophageal cancer, the promise of predicting treatment response from a routine CT scan remains just that, a promise. But by showing exactly where the promise breaks down, this proof-of-concept study may ultimately accelerate the field toward models that survive contact with honest statistics. The lesson for the radiomics community is clear: nested validation is not bureaucratic box-ticking. It is the difference between a model that works and a model that merely looks like it does.

Subject of Research: Prediction of early radiological response to radiotherapy in esophageal cancer using CT radiomics and clinical factors

Article Title: Prediction of early radiological response in esophageal cancer using CT radiomics and clinical factors: a single-center proof-of-concept study in an elderly cohort

Article References: Zhou, Q., Tao, Z.-C., Zhang, H.-B., Zhou, L.-R., & Liu, H.-W. (2026). Prediction of early radiological response in esophageal cancer using CT radiomics and clinical factors: a single-center proof-of-concept study in an elderly cohort. BMC Medical Imaging. https://doi.org/10.1186/s12880-026-02903-1

Image Credits: AI Generated

DOI: 10.1186/s12880-026-02903-1

Keywords: esophageal cancer, radiomics, machine learning, CT imaging, radiotherapy, treatment response, LASSO logistic regression, support vector machine, nested cross-validation, predictive modeling, medical imaging, elderly patients