A computational study published in Cancer Imaging demonstrates that a dual-view Vision Transformer framework, termed ExViT-Breast, significantly improves diagnostic discrimination and interpretive transparency in breast lesion classification compared to conventional convolutional neural networks and single-view models. Developed by a research team led by Ajay Kumar, the framework combines a Vision Transformer backbone with a specialized cross-view attention fusion module designed to integrate complementary diagnostic features from paired craniocaudal and mediolateral oblique mammographic projections. When evaluated on the curated Curated Breast Imaging Subset of the Digital Database for Screening Mammography, ExViT-Breast achieved an area under the receiver operating characteristic curve of 0.94 with an uncertainty margin of 0.02, significantly exceeding the performance of a standard ResNet-50 baseline and single-view transformer implementations. Beyond overall diagnostic accuracy, the study introduced a rigorous quantitative framework to evaluate explanation quality, demonstrating that Transformer-specific Layer-wise Relevance Propagation produced saliency maps with superior algorithmic faithfulness and clinical plausibility compared to widely used gradient-weighted class activation mapping techniques.
Breast cancer continues to represent the leading cause of cancer-related mortality among women worldwide, making early detection through mammographic screening vital for reducing mortality rates and improving clinical survival outcomes. While computer-aided diagnosis systems powered by deep learning—most notably convolutional neural networks—have matched or occasionally exceeded radiologist benchmarks in controlled screening tasks, their clinical translation remains constrained by inherent structural limitations. Standard convolutional architectures rely on spatially restricted receptive fields that struggle to capture long-range contextual relationships and subtle bilateral tissue asymmetries distributed across large anatomical areas. Conversely, Vision Transformers process images as sequences of discrete patches and apply global self-attention mechanisms to model interactions across the entire image field from initial layers onward. Despite these advantages, most previous transformer studies in mammography processed individual projections in isolation, disregarding the synergistic diagnostic information routinely gathered by radiologists who evaluate paired views. Furthermore, prior investigations rarely evaluated visual explanations against expert-annotated anatomical ground truth using standardized quantitative metrics.
To overcome these clinical and architectural gaps, the investigators engineered ExViT-Breast around four interconnected functional components. The pipeline begins with a patch embedding layer that segments an input image resized to 224 by 224 pixels into non-overlapping patches measuring 16 by 16 pixels, generating a structured sequence of tokens processed through twelve sequential transformer encoder blocks. For mammographic evaluations, a cross-view fusion module employs multi-head cross-attention to combine token representations extracted from corresponding craniocaudal and mediolateral oblique projections using learned attention weights, while bypassing this step when evaluating single-image ultrasound data. Classification decisions are generated by a dedicated multilayer perceptron head comprising two layers with a dropout probability of 0.1 and a sigmoid activation function applied to the fused latent representation. Running parallel to the classification pathway, a unified explainability module computes four distinct attribution maps for each diagnostic prediction, specifically comparing Grad-CAM, Grad-CAM++, LayerCAM, and Transformer-specific Layer-wise Relevance Propagation to enable direct benchmarking of interpretability techniques.
Experimental implementation and model optimization were conducted within a dedicated high-performance computing environment equipped with NVIDIA RTX A6000 graphics processing units featuring 48 gigabytes of video memory. The software pipeline utilized PyTorch 1.13.0, CUDA 11.7, and the Hugging Face Transformers library to construct and train all transformer components under uniform conditions. Hyperparameters were established through a comprehensive grid search and optimized using the AdamW algorithm configured with momentum parameters beta-one at 0.9, beta-two at 0.999, and a weight decay parameter of 0.01. Training followed a cosine annealing learning rate schedule starting from 3 times 10 to the negative fifth power, running for up to 100 epochs with an early stopping threshold set at ten validation epochs without improvement. The loss formulation incorporated binary cross-entropy with a label smoothing coefficient of 0.1, applied across mini-batches of 32 images expanded to an effective batch size of 64 through gradient accumulation steps to balance training stability with hardware memory constraints.
Model training and evaluation leveraged three established public benchmark datasets spanning mammography and ultrasound modalities, totaling thousands of thoroughly characterized patient examinations. The primary training cohort, CBIS-DDSM, comprised 1,566 subjects representing 10,239 mammographic images and 1,644 abnormal cases, partitioned at the patient level into 90 percent training and 10 percent validation splits to prevent data leakage across views. This repository exhibited balanced pathological representation, containing 886 benign cases (53.9 percent) and 758 malignant cases (46.1 percent), further categorized into 753 calcification abnormalities and 891 mass lesions accompanied by radiologist-verified bounding boxes and pixel-level segmentation contours. For external institutional validation, the team utilized the INbreast database, containing 410 high-resolution full-field digital mammograms from 115 patients recorded with 14-bit contrast precision and detailed XML annotations. To assess cross-modality generalizability, the Breast Ultrasound Images repository contributed 780 sonograms across 600 patients, partitioned at the patient level into 70 percent training, 15 percent validation, and 15 percent testing subsets across normal, benign, and malignant classes.
Standardized image preprocessing ensured uniform numeric distributions and anatomical correspondence across varied acquisition hardware prior to feature extraction. Pixel values were standardized to zero mean and unit variance computed across the training distribution, with INbreast 14-bit images undergoing linear rescaling to an eight-bit scale using metadata window center and width parameters before normalization. Mammographic view pairs underwent automated spatial registration combining global affine transformations driven by mutual information maximization and local refinement through patch-based normalized cross-correlation across 32 by 32 pixel windows, with registration quality confirmed by achieving a Dice coefficient greater than 0.85 across annotated regions. Data augmentation included horizontal and vertical reflections, rotations up to 15 degrees, random affine shifts, brightness and contrast variations of 20 percent, and modality-specific elastic deformations for mammograms alongside simulated acoustic speckle noise for sonographic images, expanding model robustness against acquisition variability.
Primary classification outcomes confirmed that ExViT-Breast delivered superior discriminative performance across the evaluated mammographic cohorts. On the primary CBIS-DDSM benchmark, the framework achieved an area under the curve of 0.94 with a 95 percent confidence interval spanning 0.92 to 0.96. Statistical hypothesis testing via the DeLong method confirmed this performance was significantly higher than the 0.89 area under the curve recorded by a standard ResNet-50 convolutional network (p less than 0.001) and the 0.91 achieved by an equivalent single-view Vision Transformer (p equals 0.003). When tested without retraining on the external INbreast cohort, the architecture demonstrated strong domain transferability, yielding an area under the curve of 0.91 with a standard deviation of 0.03 while preserving a high clinical sensitivity of 88.7 percent at the strict operational threshold of one false positive per patient examination.
Cross-modality transfer to the ultrasound domain revealed notable performance differences that illustrate the boundaries of mammography-trained representations when exposed to distinct imaging physics. On the BUSI ultrasound dataset, ExViT-Breast achieved an area under the curve of 0.88 with a standard deviation of 0.04 and a 95 percent confidence interval from 0.84 to 0.92, representing a statistically significant decline of 0.06 compared to primary mammographic performance (p equals 0.011) and a non-significant drop of 0.03 relative to INbreast (p equals 0.09). Subgroup analysis indicated that the model classified malignant lesions with higher accuracy, reaching an area under the curve of 0.91, compared to differentiating normal acoustic tissue, which recorded an area under the curve of 0.85. The authors attributed this divergence to fundamental disparities between X-ray transmission and acoustic wave reflection, differences in tissue contrast mechanisms, probe handling variations, and the necessary omission of cross-view fusion for single-view sonographic data.
A critical contribution of the research was the systematic benchmarking of visual attribution techniques using objective mathematical definitions of explanation faithfulness, clinical plausibility, and uncertainty. Faithfulness was quantified through deletion and insertion curves, measuring prediction degradation as salient image patches were progressively masked or revealed. Transformer-LRP achieved a faithfulness score of 0.71 on CBIS-DDSM, significantly outperforming conventional Grad-CAM, which achieved 0.62 (p less than 0.001). Clinical plausibility, measured via pointing game accuracy against expert lesion masks, revealed that 79.5 percent of peak activations generated by Transformer-LRP fell strictly inside expert-annotated boundaries, compared to 71.4 percent for Grad-CAM (p equals 0.002), with malignant lesions displaying higher localization accuracy (82.3 percent) than benign findings (76.8 percent). Furthermore, ensemble-based uncertainty quantification showed that Transformer-LRP exhibited 31 percent lower variance than Grad-CAM, with low explanation uncertainty strongly correlating with correct diagnostic classifications (r equals negative 0.68, p less than 0.001).
Component ablation experiments and operational runtime profiling highlighted the practical strengths and clinical boundaries of the proposed diagnostic architecture. Removing cross-view fusion caused a statistically significant drop in performance (delta area under the curve of negative 0.02, p equals 0.04), diminishing specificity by 3.2 percent, sensitivity by 1.8 percent, and plausibility by 4.4 percentage points. Pretraining initialization with self-supervised DINOv2 matched ImageNet-22k classification performance while enhancing plausibility to 80.2 percent (p equals 0.03). At inference on RTX A6000 hardware, single-image classification required 18.3 milliseconds, while generating four simultaneous explanations required 45.7 milliseconds, with batch processing of 32 images executing in 287 milliseconds. However, error analysis showed that 62 percent of incorrect predictions occurred in dense fibroglandular tissue (BI-RADS categories C and D) and 28 percent involved subtle architectural distortions. The authors explicitly emphasized that the investigation represents a foundational computational study without radiologist reader trials, underscoring that prospective clinical validation and formal expert evaluation remain essential before real-world deployment.
Subject of Research: Explainable Vision Transformer architecture for multi-view breast lesion classification
Article Title: Explainable vision transformer framework for breast lesion classification with Grad-CAM support
Article References: Kumar, A., Khan, M., Mittal, G., Husain, I., Shukla, S., Ali, S. A., Sayal, A., Jha, J., & Ziar, R. A. (2026). Explainable vision transformer framework for breast lesion classification with Grad-CAM support. Cancer Imaging, 26(1), Article 150. https://doi.org/10.1186/s40644-026-01097-7
Image Credits: AI Generated
DOI: 10.1186/s40644-026-01097-7
Keywords: Vision Transformers, Breast Cancer Detection, Mammography, Explainable AI, Transformer-LRP, Grad-CAM, Cross-View Attention, CBIS-DDSM, INbreast, BUSI Dataset, Ultrasound Imaging, Computer-Aided Diagnosis, Medical Image Analysis, Deep Learning, Model Calibration, Saliency Mapping, Ablation Study
News Source: Ophelia Keating. (November 11, 2026). Explainable Vision Transformer Framework Advances Multi-View Breast Lesion Classification. Scienmag.
