Attention-driven multimodal AI shows promise for lung cancer diagnosis, review finds

Lung cancer remains the deadliest cancer in the world, responsible for nearly 1.8 million of the roughly 2.48 million new cases recorded in 2022, and a new systematic review argues that the most promising path toward better diagnosis lies in artificial intelligence that can look at many kinds of evidence at once. The review, published in Discover Artificial Intelligence, systematically analyzed 43 studies published between 2020 and 2025 that combine multiple data types—medical imaging, genomic profiles, and clinical records—with attention-based deep learning. Its central conclusion is that multimodal deep learning consistently outperforms single-modality approaches across the major lung cancer diagnostic tasks, with reported gains ranging from a few percentage points in binary detection to an absolute improvement of 0.19 AUC in predicting immunotherapy response.

The review, led by Rumeth Payagalage, Nipuna Senanayake, and Prasan Yapa, applied PRISMA-style screening to records drawn from Google Scholar, PubMed, and IEEE Xplore. After removing duplicates and excluding studies limited to a single data modality, studies not focused on lung cancer, and studies lacking an attention mechanism, 43 studies remained for analysis. The authors organized the literature along a five-dimensional taxonomy covering data modality, clinical task, backbone architecture, fusion strategy, and attention mechanism. They also performed what they describe as the first analysis of hyperparameter configurations across all five clinical tasks in this space, and identified open challenges that stand between laboratory performance and clinical deployment.

The case for multimodality rests on the complementary nature of the data sources. Computed tomography provides detailed anatomical structure at sub-millimeter resolution, encoding tissue density in Hounsfield Units, but reveals nothing about molecular activity. Positron emission tomography captures metabolic activity through the accumulation of fluorodeoxyglucose in metabolically active tumor cells, highlighting malignant lesions as hotspots, yet its spatial resolution of roughly four millimeters is far coarser than CT’s. Whole slide images—digitized histopathology slides that can reach 150,000 by 150,000 pixels—reveal cellular morphology and tissue architecture at subcellular resolution. Genomic data such as RNA sequencing, copy number variation, and somatic mutation profiles describe tumor biology without any spatial context, while structured clinical records supply prognostic history that neither imaging nor genomics can capture.

Attention mechanisms, first proposed for sequence-to-sequence learning and later generalized in the Transformer architecture, are the technical thread running through all 43 studies. The review distinguishes three levels at which attention operates. Within individual modalities, self-attention and spatial or channel attention suppress noise and highlight diagnostically relevant features. Across modalities, cross-modal attention aligns heterogeneous representations and captures interactions between genotype and phenotype—for example, using genomic features to guide the selection of diagnostically important regions within a whole slide image. At the level of entire data sources, modality-level attention dynamically weights each modality’s contribution based on the clinical task and data availability, which also allows models to cope when a data type is missing.

The clinical tasks covered span the full diagnostic pathway: lung tumor detection, subtype classification, tumor segmentation, survival prediction, treatment response forecasting, and post-surgical recurrence monitoring. Each task has its own dominant data combination and evaluation metric. Detection relies heavily on CT paired with clinical biomarkers or PET, prioritizing sensitivity and false positives per scan. Subtype classification—distinguishing adenocarcinoma from squamous cell carcinoma, for instance—pairs imaging with genomics and is judged primarily by AUC. Segmentation, critical for radiotherapy planning, favors PET/CT and is measured with the Dice Similarity Coefficient and the 95th percentile Hausdorff distance. Survival prediction typically combines whole slide images with genomic data and is evaluated with Harrell’s concordance index.

Several reported results illustrate the scale of the gains. In survival prediction, a co-attention model combining whole slide images with genomic data achieved a mean C-index of 0.722 on the TCGA-LUAD cohort, a 5.7 percent improvement over a concatenation-based baseline, and late fusion of separately trained models reached only 0.682 on the same cohort—a four percentage point gap attributed to joint cross-modal interaction learning. In segmentation, PET-guided spatial attention improved gross tumor volume delineation by 7.6 percent in Dice score over a CT-only baseline. In treatment response prediction, integrating CT, PD-L1 immunohistochemistry slides, and genomics raised AUC to 0.80, an absolute gain of 0.19 over tumor mutational burden alone and 0.07 over PD-L1 score alone. A recurrence prediction model trained on longitudinal CT plus 64 clinical and molecular features reached an AUC of 0.854 across 14,177 patients.

The choice of backbone architecture matters as much as the data. Convolutional neural networks remain the most common choice, prized for local spatial feature extraction, computational efficiency, and the ability to work with small datasets, though their receptive field stays local no matter how deep the network. Transformers excel at global context modeling and at representing heterogeneous data types as tokens in a shared space, but their quadratic self-attention cost makes whole slide images—with up to 230,000 patches—computationally prohibitive without approximation methods, and they tend to overfit on small cohorts. Graph neural networks explicitly model spatial topology, preserving neighborhood relationships between tissue regions that attention-based multiple instance learning treats as unordered bags of patches. Hybrid and ensemble architectures, combining convolutional and transformer components or training independent modality-specific models with a meta-learner, achieved some of the highest reported scores, including an AUC of 0.99 in a four-level risk classification task that combined imaging, sensor streams, and electronic records.

Fusion strategy emerged as a decisive design choice. Early fusion, which concatenates raw inputs before feature extraction, is simple and efficient but demands pixel-level alignment that only hardware-registered PET/CT can reliably provide. Late fusion, which trains independent models and combines their predictions, is robust to missing modalities but cannot learn feature-level cross-modal interactions. Feature-level fusion, the most common strategy in the reviewed studies, combines learned representations in a shared latent space and, when guided by attention, produced the highest performance across tasks. Hybrid approaches layering multiple fusion stages consistently outperformed single-stage alternatives, though at higher computational cost and greater risk of training instability.

The review is candid about the obstacles separating these models from routine clinical use. Most models require every modality they were trained on at inference time, an unrealistic demand given that PET scanners, genomic testing, and complete records are not universally available; only a handful of studies tested missing-modality robustness, with one retaining an AUC of 0.73 when genomics were withheld. Most studies also relied on a single public dataset, raising serious questions about generalization across institutions and scanners. The authors caution that attention weights, the most common interpretability tool, should not be treated as faithful explanations, since high attention scores do not reliably indicate causal importance, and none of the reviewed studies validated attention maps against independent radiologist annotations in a formal user study. Rare subtypes accounting for less than five percent of cases remain difficult to classify, and only one reviewed study modeled tumor change over time longitudinally.

The authors also analyzed hyperparameter patterns that could lower the cost of developing future models. Survival prediction studies predominantly used a batch size of one, a consequence of multiple instance learning over whole slide images, with effective batch sizes of 16 to 32 achieved through gradient accumulation. Treatment response studies, working with cohorts as small as 42 patients, required higher dropout rates of 0.3 to 0.5, learning rates as low as 0.00001, and nested cross-validation to prevent overfitting. Three-dimensional CT-based detection and segmentation models were constrained to small batches by GPU memory, creating a trade-off between patch size and batch size. The review concludes that if missing-modality robustness, multi-site external validation, mechanistic interpretability, and standardized evaluation protocols can be addressed, attention-based multimodal deep learning could contribute meaningfully to accurate, non-invasive, and interpretable lung cancer diagnosis.

Subject of Research: Attention-based multimodal deep learning techniques for lung cancer diagnosis

Article Title: A comprehensive review of attention-based multimodal deep learning techniques for lung cancer diagnosis

Article References: Payagalage, R., Senanayake, N., & Yapa, P. (2026). A comprehensive review of attention-based multimodal deep learning techniques for lung cancer diagnosis. Discover Artificial Intelligence, 6(1), Article 1652. https://doi.org/10.1007/s44163-026-02496-2

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02496-2

Keywords: lung cancer, multimodal deep learning, attention mechanisms, medical imaging, genomics, survival prediction, tumor segmentation, fusion strategies, systematic review, clinical deployment, interpretability, NSCLC