Dual-branch AI fuses local and global MRI features to sort brain tumors with 99% accuracy

A new deep learning model called NeuroAttnFuseNet has achieved 99.24% accuracy on a fixed hold-out test set of 1,311 brain MRI images, correctly classifying scans into four categories—glioma, meningioma, pituitary tumor, and no tumor—with only 10 misclassifications out of 1,311 predictions. The study, published open access in Discover Artificial Intelligence, describes a dual-path architecture that combines two complementary ways of seeing an image: one branch that excels at fine-grained local detail and another that captures the global context of the whole scan. The researchers report that this attention-guided fusion of local and global features outperformed six widely used baseline models under an identical evaluation protocol, though they are careful to stress that the results are image-level evidence within a merged public dataset, not proof of clinical generalization.

The motivation for the work lies in a persistent problem in automated brain tumor classification: tumor appearance varies enormously across patients and imaging conditions, while the categories that matter clinically can differ by subtle cues rather than a single dominant visual signature. Tumors within the same class can differ in location, size, boundary sharpness, intensity profile, and surrounding edema patterns, and lesions in different classes can occupy similar anatomical regions or share similar intensity distributions, increasing the likelihood of confusion between categories such as glioma and meningioma. On top of this, acquisition variability—changes in contrast, noise, or scan orientation—complicates consistent feature learning, and labeled MRI datasets are often small relative to the capacity of modern models, encouraging overfitting.

Perhaps the most insidious challenge the authors highlight is data leakage. Public 2D MRI datasets that are merged from multiple sources frequently contain duplicate or near-duplicate images, and when such redundant images inadvertently appear in both training and test splits, reported performance can be artificially inflated. The team therefore built leakage control into the evaluation design as a central requirement rather than an afterthought. Exact duplicates were detected using file-level hashing and pixel-array consistency checks, while near-duplicates—images that differ only slightly due to resizing, compression, or minor intensity shifts—were identified using perceptual hashing (pHash), which is sensitive to visual resemblance rather than byte equality. Screening was performed after merging the three source datasets and before finalizing the split, and redundant samples were removed or confined to a single partition.

The dataset itself was assembled by combining three public collections—Figshare, SARTAJ, and Br35H—yielding 7,023 MRI images labeled as glioma, meningioma, pituitary, or no tumor. After harmonizing labels and removing unreadable or corrupted files, the researchers established a leakage-safe hold-out split of 5,712 training images (81.33%) and 1,311 test images (18.67%). The training class distribution was moderately imbalanced, with the no tumor class the largest in both partitions. Augmentation—random rotation within ±15 degrees, horizontal flipping, shifts up to 10%, zoom between 0.9 and 1.1, and brightness and contrast adjustments within ±10%—was applied only to training data and only after the split was finalized. Vertical flips were deliberately excluded because they can produce anatomically implausible brain orientations. The test set underwent only deterministic formatting such as resizing to 224 × 224 and normalization to the [0,1] range, and was never accessed during model development except for final evaluation.

At the heart of the study is the architecture itself. NeuroAttnFuseNet processes each MRI through two parallel branches with complementary roles. The first is a Swin Transformer feature extractor that learns hierarchical, multi-scale local morphological features—boundary irregularities, texture granularity, and localized structural patterns relevant to tumor characterization. The second is a pretrained DINOv2 ViT-B/14 encoder, kept frozen and used strictly as a feature extractor, which converts the input into patch tokens and aggregates them into a 768-dimensional global descriptor derived from the [CLS] token. This global embedding encodes long-range dependencies and holistic context that can remain stable under limited labeled data and appearance variation. The two feature vectors—768 dimensions from each branch—are concatenated into a 1,536-dimensional fused embedding.

What distinguishes the approach from naive fusion is the channel-attention gate that follows concatenation. A lightweight gating network models inter-channel dependencies and outputs normalized channel-wise weights, computed from the fused representation itself, which are applied via elementwise multiplication to amplify informative channels and suppress less relevant ones. The authors argue this addresses a known weakness of simple concatenation: when one branch dominates due to scale differences, noise sensitivity, or dataset-specific bias, the fused model can behave like a single-branch baseline. With attention-guided gating, the network can adapt its reliance on local morphology versus global context on a per-sample basis—emphasizing fine detail when it is decisive and broader context when morphology is ambiguous. The gated representation then passes through a fusion MLP and a final softmax classifier. The full model contains approximately 115.68 million parameters, of which only 29.10 million are trainable because the DINOv2 branch is frozen.

The performance results are striking, though reported with appropriate nuance. On the fixed hold-out test set, the model achieved accuracy 0.9924, macro precision 0.9922, macro recall 0.9917, macro F1 0.9919, macro AUC 0.9991, and macro average precision 0.9969. The confusion matrix shows 1,301 correct predictions and 10 errors, with the no tumor class perfectly recognized at 405 out of 405. The residual errors were concentrated in subtle tumor-to-tumor distinctions: glioma accounted for 4 errors (2 predicted as meningioma, 2 as pituitary), meningioma for 4 errors (2 as no tumor, 2 as pituitary), and pituitary for 2 errors. Under repeated five-fold evaluation with three random seeds—producing 15 paired run-level comparisons per baseline—the model achieved 99.07 ± 0.10% accuracy, ahead of CoAtNet (98.80 ± 0.12%), ConvNeXt (98.75 ± 0.13%), Swin Transformer (98.45 ± 0.10%), InceptionV3 (98.34 ± 0.20%), ResNet50 (98.18 ± 0.16%), and MobileNetV2 (98.15 ± 0.18%). Paired t-tests across folds yielded p < 0.001 for each comparison, though the authors note that the Wilcoxon signed-rank test produced p = 0.0625 throughout, which does not meet the 0.05 threshold and warrants conservative interpretation of statistical significance.

An ablation study isolated the contribution of the fusion strategy itself, comparing single-branch models, simple concatenation, MLP-based fusion, cross-attention fusion, and the proposed channel-attention fusion. The channel-attention configuration achieved the highest five-fold cross-validation accuracy, hold-out accuracy, macro F1, and macro AUC, improving five-fold accuracy by 0.26 percentage points over concatenation with MLP fusion—a modest but consistent gain attributable to the gating mechanism. Supplementary analyses added further texture to the picture. Test-time perturbation experiments showed the largest performance reductions under severe downsampling and large rotations, indicating sensitivity to strong spatial degradation. Calibration metrics, including low expected calibration error and Brier score, suggested that predicted probabilities were well aligned with actual correctness, and that incorrect predictions tended to receive lower confidence—though this observation rests on only 10 errors. A detailed audit of all ten misclassified cases showed that the true class remained the second-ranked prediction in most instances.

The authors are notably candid about limitations, and their caveats deserve as much attention as the headline numbers. Patient identifiers were unavailable for the merged dataset, so correlations among slices from the same patient, repeated acquisitions, or source-specific curation patterns cannot be fully excluded; the results therefore represent image-level evidence, not patient-level or external clinical generalization. Source-stratified analysis revealed that performance was not uniform across the three source datasets, with lower performance on the SARTAJ subset, and because Br35H contributes only no-tumor images, source identity and class identity are partially confounded. MRI sequence metadata was heterogeneous and incomplete, meaning the model may partly learn dataset-specific visual cues rather than tumor morphology alone. The model also operates on individual 2D images without volumetric context, and its computational footprint is substantial—more parameters, FLOPs, memory, and inference time than the lightweight baselines—positioning it as an accuracy-oriented research model rather than a deployment-ready clinical system.

The study’s broader significance may lie less in the specific accuracy figure than in its methodological stance. By pairing a local-global fusion architecture with explicit duplicate screening, training-only augmentation, and strict test isolation, the work offers a template for how near-perfect results on merged public benchmarks should be earned and reported. The authors’ future directions—patient-level splitting, independent multi-center validation across institutions and scanners, dataset-wise generalization testing, 3D volumetric modeling, and uncertainty-aware prediction—map out the path such models must travel before any claim of clinical utility could be justified. For now, NeuroAttnFuseNet stands as a carefully evaluated demonstration that combining hierarchical local morphology with pretrained global context, balanced through learned attention, can push four-class brain MRI classification to the ceiling of what a rigorously screened public dataset can measure.

Subject of Research: Dual-branch attention fusion deep learning for four-class brain tumor MRI classification

Article Title: NeuroAttnFuseNet dual branch attention fusion for brain tumor MRI classification

Article References: Sabuj, M. S. H., Sultana, F., Rashid, M. H. O., Mullela, R. M., & Yusuf, M. (2026). NeuroAttnFuseNet dual branch attention fusion for brain tumor MRI classification. Discover Artificial Intelligence, 6(1), Article 964. https://doi.org/10.1007/s44163-026-01995-6

Image Credits: AI Generated

DOI: 10.1007/s44163-026-01995-6

Keywords: brain tumor, MRI classification, deep learning, Swin Transformer, DINOv2, attention fusion, data leakage, glioma, meningioma, pituitary tumor, computer vision, medical imaging