ResNet50 emerges as most efficient AI backbone for label-free IBD histopathology

Artificial intelligence models trained to detect inflammatory bowel disease from label-free microscopy images perform best when built on ResNet50 with partial fine-tuning and data augmentation, according to a systematic comparison of nine convolutional neural network architectures published in the journal Discover Imaging.

The research addresses a practical bottleneck in inflammatory bowel disease care. Assessing the activity of Crohn’s disease and ulcerative colitis currently depends on biopsy processing and manual review of stained tissue sections by pathologists, a time-consuming process that limits real-time clinical use. The authors point to multimodal nonlinear optical microscopy as an alternative: by combining coherent anti-Stokes Raman scattering, two-photon excited autofluorescence, and second-harmonic generation, the technique produces stain-free images of colonic tissue at subcellular resolution, visualising cellular membranes, metabolic activity, and collagen remodelling directly.

Earlier computational analyses of such multimodal images relied on classical machine learning pipelines that required manually annotated masks and handcrafted features such as crypt area, equivalent diameter, and intensity statistics, followed by ranking methods and a linear classifier. The authors argue that this dependence on manual input creates a scalability bottleneck, generalises poorly across patients and scanners, and fails to capture complex spatial dependencies. Deep learning can learn discriminative features directly from raw images, but biomedical datasets are typically small, raising the question of whether transfer learning from natural-image pretraining helps or hinders in this unusual imaging domain.

To answer that question systematically, the team evaluated nine CNN architectures—VGG16, InceptionV3, ResNet50, ResNet101, ResNet152, Xception, DenseNet121, MobileNet, and EfficientNetB0—under three training regimes: partial fine-tuning, in which only the upper fraction of pretrained backbone layers was unfrozen; full fine-tuning, in which the entire pretrained backbone was retrained; and training from scratch with random initialisation. Each configuration was tested with and without patch-level augmentation, and performance was assessed at both the patch and patient levels using 20-fold patient-level cross-validation, with uncertainty quantified through bootstrap confidence intervals based on 5000 resamples.

The underlying data came from 20 patients with Crohn’s disease, ulcerative colitis, or infectious colitis, whose colonic biopsies were imaged without staining. Rather than using diagnostic categories directly, the researchers combined three histopathological labels—tissue architecture, chronicity, and inflammatory activity—into a composite inflammation score ranging from 0 to 5, with scores above 3 defining the diseased class. This yielded 13 patients in the low-inflammation class and 7 in the diseased class, from which 792 image patches were extracted from large mosaic scans. Augmentation, applied only to training data to prevent leakage, generated three additional variants per patch using flips, small affine transformations, brightness and contrast adjustments, Gaussian noise, and colour shifts.

With augmented training data, the strongest architecture dependence appeared under partial fine-tuning. All ResNet variants formed a clearly separated top group with tightly clustered results across validation folds, significantly outperforming every non-ResNet architecture. Under this restricted adaptation regime, MobileNet, EfficientNetB0, and DenseNet121 showed broad, unstable distributions that varied strongly with the held-out patient. At the patient level, all three ResNet models correctly classified all 20 patients, while MobileNet managed 16 of 20 and DenseNet121 only 14 of 20, showing how fold-level instability translates into errors when adaptation capacity is limited.

Perhaps the most striking finding concerns what happens when pretraining is abandoned altogether. Many models trained from scratch matched or exceeded their transfer-learning counterparts, which the authors interpret as evidence that ImageNet features do not always align well with multimodal microscopy data. Natural-image pretraining encodes object-centric layouts and RGB colour semantics, whereas CARS, TPEF, and SHG images are dominated by modality-specific physical contrast and texture-like statistics without discrete objects. DenseNet121, with only about 7 million parameters, achieved near-perfect accuracy even from random initialisation, which the authors attribute to its dense connectivity: because each layer receives all preceding feature maps, gradient flow and feature reuse remain efficient even when the target domain differs sharply from natural images.

The efficiency analysis reinforced the case for ResNet50. With roughly 14.5 million trainable parameters under partial fine-tuning, it reached about 97 percent patch-level balanced accuracy and 100 percent patient-level accuracy, while scaling up to ResNet101 (about 42 million parameters) and ResNet152 (about 58 million) brought no measurable gain, with accuracies plateauing around 95 to 97 percent despite a three- to fourfold increase in parameters. Lightweight models told a different story: EfficientNetB0 and InceptionV3 failed to converge from scratch without augmentation, reaching only about 41 and 65 percent balanced accuracy respectively, but augmentation lifted them to roughly 94 and 96 percent. MobileNet and Xception were similarly unstable under partial fine-tuning without augmentation, at 65.2 and 71.2 percent balanced accuracy, and recovered only with fuller retraining.

Augmentation itself proved to be architecture-dependent rather than universally beneficial. Under partial and full fine-tuning it generally improved accuracy and reduced fold-to-fold variability, with the largest full fine-tuning gains for EfficientNetB0 (+14 percentage points), DenseNet121 (+9), and MobileNet (+8). Under training from scratch, however, the picture inverted: augmentation produced enormous gains for the weakest models—+55 percentage points for EfficientNetB0 and +31 for InceptionV3—yet significantly degraded all ResNet models, with mean accuracy dropping by 3 to 9 percentage points and variability rising. The authors suggest that for high-capacity backbones without pretrained initialisation, the added variability may destabilise optimisation. They caution that it remains unclear whether augmentation gains stem from increased effective sample size or from specific transformations, a question they flag for future work.

The authors are careful to frame the study’s scope and limitations. The cohort was modest—20 patients from a single imaging protocol described in earlier work by Chernavskaia and colleagues—and only ImageNet pretraining was considered, leaving domain-specific pretraining on large multimodal histology datasets untested. The analysis was restricted to standard CNN backbones, excluding transformers, graph-based models, and explicit multimodal fusion methods. Only one representative channel per modality was used to reduce redundancy. Within those limits, the practical recommendation is clear: for transfer-learning setups in multimodal IBD histopathology, ResNet50 with partial fine-tuning and augmentation provides an efficient and robust baseline, while suitably connected architectures such as DenseNet121 can still learn effectively from scratch on limited datasets—meaning architecture choice, adaptation capacity, and augmentation must be considered jointly rather than in isolation.

Subject of Research: Comparing transfer learning strategies of nine CNN architectures for classifying inflammatory bowel disease from label-free multimodal microscopy images

Article Title: Comparative analysis of transfer learning architectures for multimodal microscopy based IBD histopathology

Article References: Kamran, J., Meyer-Zedler, T., Schmitt, M., Waldner, M. J., Vieth, M., Popp, J., & Bocklitz, T. (2026). Comparative analysis of transfer learning architectures for multimodal microscopy based IBD histopathology. Discover Imaging, 3(1), Article 2. https://doi.org/10.1007/s44352-026-00024-7

Image Credits: AI Generated

DOI: 10.1007/s44352-026-00024-7

Keywords: inflammatory bowel disease, transfer learning, convolutional neural networks, multimodal microscopy, CARS, TPEF, SHG, histopathology, data augmentation, ResNet50, DenseNet121, label-free imaging