AI Combines MRI Scans and Tissue Slides to Predict Chemotherapy Response in Nasopharyngeal Cancer

For patients with locally advanced nasopharyngeal carcinoma, one of the most consequential moments in treatment arrives before therapy even begins. Doctors must decide whether induction chemotherapy, given before the standard combination of chemoradiotherapy, is likely to shrink the tumor enough to justify its burdens. That decision currently rests on coarse tools: imaging read by eye, staging systems refined over decades, and clinical judgment. A new study published in PLOS Digital Health suggests that artificial intelligence can do considerably better, by teaching a single deep learning model to read both a patient’s MRI scans and the microscopic architecture of their tumor tissue at the same time.

The research team, led by Jing Hou, Xiaochun Yi, Congrui Li, Junjian Li, Hui Cao, Qiang Lu, and Xiaoping Yu, assembled pretreatment MRI scans and digitized tissue slides, known as whole slide images, from 404 patients with locally advanced nasopharyngeal carcinoma. Their goal was ambitious: to build a single model, called MoEMIL, that could simultaneously predict two distinct outcomes. The first is how well a patient responds to induction chemotherapy in its early phase, and the second is how long that patient is likely to survive overall. In clinical practice these two questions are usually answered separately, if they are answered quantitatively at all, and the researchers bet that the information needed to answer them overlaps in ways a joint model could exploit.

The technical heart of the study lies in how MoEMIL handles two radically different kinds of medical data. An MRI scan is a three-dimensional volumetric image with a fixed, orderly structure, and the researchers extracted features from it in two complementary ways. One pipeline used PyRadiomics, a widely adopted open-source toolkit that computes hand-crafted quantitative descriptors of image texture, shape, and intensity. The other pipeline fed the scans through ResNet50, a convolutional neural network that learns its own visual features from the data rather than relying on predefined measurements. Running both approaches in parallel allowed the model to capture statistical regularities that radiologists have long described qualitatively, alongside subtle patterns that no human eye is trained to notice.

Whole slide images posed a very different computational challenge. A single digitized pathology slide can contain billions of pixels, far too many for a neural network to process in one pass, and the diagnostically relevant cells may occupy only a tiny fraction of the slide. The standard solution, which MoEMIL adopts, is multiple instance learning. The slide is cut into thousands of small patches, each patch is encoded individually, and an attention mechanism learns to weight the patches according to how informative they are for the prediction. In effect, the model teaches itself which regions of the tumor matter, without anyone having to annotate them. The researchers employed a variant known as clustering-constrained attention multiple instance learning, which organizes patches into groups to sharpen this focus.

With features flowing in from both imaging modalities, the model needed a way to combine them. MoEMIL uses a multi-gate mixture-of-experts architecture, a design in which several specialized subnetworks, the experts, each process the fused multimodal information, and learned gating networks decide how much each expert should contribute to each task. The key advantage is that the two prediction tasks, chemotherapy response and overall survival, share some underlying biology but not all of it. The gating mechanism lets the model route shared knowledge where it helps and keep task-specific reasoning separate where it does not, avoiding the trap of forcing one compromise prediction to serve two different clinical questions.

Interpretability was treated as a requirement rather than an afterthought. Deep learning models in medicine face a persistent credibility problem: they often outperform traditional methods while offering no explanation for their conclusions. The researchers applied gradient-weighted class activation mapping to highlight which regions of the MRI scans drove the model’s predictions, and used the attention weights from the multiple instance learning pipeline to indicate which patches on the pathology slides were most influential. These visualization techniques do not constitute a full causal explanation, but they give clinicians a way to inspect what the model is looking at and to check whether its attention lands on biologically plausible structures rather than imaging artifacts.

The performance results are striking. In the task of distinguishing good responders from poor responders to induction chemotherapy, MoEMIL achieved an area under the curve of 0.917 in the training set, 0.869 in the validation set, and 0.801 in the independent test set. An area under the curve of 0.801 on unseen patients indicates genuinely useful discriminative power, well above what TNM staging, the standard anatomical classification system, achieves for this purpose. In head-to-head comparisons on the test set, MoEMIL significantly outperformed both a deep learning radiomics model built from imaging alone and conventional TNM staging for both endpoints, suggesting that the multimodal fusion is doing real work rather than merely repackaging existing information.

The comparison against a pathology-only model is particularly revealing. When the researchers pitted MoEMIL against a model that saw only the whole slide images, the multimodal approach achieved a significant improvement in predicting overall survival, with a P value of 0.0049, and a borderline improvement in predicting chemotherapy response, with a P value of 0.064. The pattern implies that the MRI and the pathology slide carry partly complementary information: the scan captures the tumor’s relationship to surrounding anatomy and its internal texture at macroscopic scale, while the slide reveals cellular and tissue-level organization that no scan can resolve. Survival prediction, it appears, benefits most from having both views available.

For the survival endpoint specifically, MoEMIL stratified patients into high-risk and low-risk groups with statistically significant separation, at P values below 0.05. This kind of risk stratification matters because it could, in principle, guide treatment intensity. Patients flagged as low risk might be spared the most aggressive regimens and their associated toxicity, while high-risk patients could be candidates for escalated therapy or closer surveillance. One notable feature of the cohort deserves mention: no cases of progressive disease were observed during induction chemotherapy, so the poor responder group in this study consisted entirely of patients with stable disease. That constraint shapes how the response prediction should be interpreted, since the model was effectively distinguishing patients whose tumors shrank from those whose tumors merely held still.

The authors position MoEMIL as an adjunctive tool rather than a replacement for clinical judgment, and that framing is appropriate for what the evidence shows. The study is retrospective, drawing on patients already treated, and the model’s predictions would need prospective validation before influencing real treatment decisions. Still, the work demonstrates a template that is likely to spread across oncology: rather than building separate models for imaging, pathology, and each clinical endpoint, a multi-task multimodal architecture can learn from everything at once and, in this case, predict both early treatment response and long-term survival with accuracy that exceeds the tools clinicians currently have. For a cancer that is particularly common in parts of East and Southeast Asia, where nasopharyngeal carcinoma imposes a heavy disease burden, a reliable early signal of whether induction chemotherapy is working could meaningfully change how treatment is planned, monitored, and personalized.

Subject of Research: Multi-task deep learning for predicting induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma

Article Title: Multi-task deep learning integrating pretreatment MRI and whole slide images predicts induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma

Article References: Hou, J., Yi, X., Li, C., Li, J., Cao, H., Lu, Q., & Yu, X. (2026). Multi-task deep learning integrating pretreatment MRI and whole slide images predicts induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma. PLOS Digital Health, 5(10), e0001378. https://doi.org/10.1371/journal.pdig.0001378

Image Credits: AI Generated

DOI: 10.1371/journal.pdig.0001378

Keywords: nasopharyngeal carcinoma, deep learning, multi-task learning, MRI, whole slide images, induction chemotherapy, survival prediction, radiomics, multiple instance learning, multimodal fusion, pathomics, prognostic stratification