A systematic empirical investigation has revealed that performance gains achieved by deep convolutional neural networks pretrained on ImageNet do not translate proportionally to medical image classification tasks, with transfer learning efficacy differing substantially across medical imaging modalities. Evaluating model behavior across distinct clinical diagnostic domains, researchers found that the effectiveness of transferring pretrained features depends heavily on the specific imaging modality, the extent of domain shift, and the chosen training strategy rather than model complexity alone. In particular, fixed feature extraction produced stark performance disparities between modalities, while comprehensive model fine-tuning consistently elevated diagnostic accuracy across all tested clinical data types.
The empirical study examined three distinct imaging modalities that serve as primary screening and detection tools for various pulmonary and ophthalmic conditions: chest radiographs, optical coherence tomography, and whole slide histopathology images. To benchmark model performance systematically across binary and multiclass tasks, the investigators evaluated ten convolutional neural network architectures across three publicly available medical imaging datasets. For binary classification of pulmonary pathology, the evaluation utilized a chest X-ray pneumonia dataset divided into 4,000 training images, 1,043 validation images, and 624 testing images. Multiclass diagnostic evaluations were conducted using an optical coherence tomography retinal dataset containing 86,647 training images, 21,662 validation images, and 1,000 testing images, alongside the LC25000 histopathology whole slide image dataset comprising 9,600 training images, 1,680 validation images, and 720 testing images.
To understand how different training paradigms influence diagnostic classification, the researchers evaluated the architectures under five distinct training strategies. These experimental configurations included three transfer learning workflows initialized with ImageNet pretraining, consisting of frozen feature extraction, partial fine-tuning, and full fine-tuning, as well as two randomly initialized baseline setups trained from scratch, consisting of partial training and full training from scratch. The classification capabilities of the models were comprehensively assessed across the datasets using four standard performance metrics: area under the receiver operating characteristic curve, precision, sensitivity, and specificity. To ensure rigorous statistical evaluation and capture experimental uncertainty, all performance metrics were reported as the mean ± standard deviation alongside their corresponding 95% confidence intervals.
The experimental results demonstrated pronounced differences in how individual imaging modalities respond to transfer learning configurations, particularly when model weights remain fixed. Under the frozen feature extraction strategy, models achieved substantially lower classification performance on both chest X-ray and optical coherence tomography datasets, where area under the receiver operating characteristic curve values fell to between 0.64 and 0.88. In contrast, whole slide histopathology classification in the LC25000 dataset benefited notably from ImageNet features even under frozen feature extraction, maintaining area under the curve values above 0.96. This performance level gave histopathology whole slide images an approximate 20% advantage in area under the curve compared to chest X-ray and retinal optical coherence tomography under identical frozen feature conditions.
Transitioning from frozen representations to full fine-tuning consistently boosted diagnostic performance across all three clinical modalities. With complete fine-tuning, the top-performing model configurations achieved area under the curve metrics of 0.9916 for chest X-ray pneumonia classification, 0.9701 for optical coherence tomography retinal classification, and 0.9980 for whole slide histopathology classification. Crucially, the researchers observed that larger and more complex convolutional architectures did not consistently outperform smaller neural network architectures, defying common assumptions regarding parameter scale. The authors concluded that the optimal pairing between a deep learning model and an imaging modality is dictated by specific dataset characteristics, domain shift, and training strategy, offering practical insights for engineering transfer learning pipelines in clinical image analysis.
Subject of Research: Transfer learning performance of pretrained convolutional neural networks across medical imaging modalities
Article Title: Transfer learning and impact of pretrained deep convolution neural networks on medical image classification: an empirical evaluation with diverse imaging modalities
Article References: Borah, J., & Singh, H. K. (2026). Transfer learning and impact of pretrained deep convolution neural networks on medical image classification: an empirical evaluation with diverse imaging modalities. Neural Computing and Applications, 38(21), Article 874. https://doi.org/10.1007/s00521-026-12585-z
Image Credits: AI Generated
DOI: 10.1007/s00521-026-12585-z
Keywords: transfer learning, deep learning, medical image classification, convolutional neural networks, chest X-ray, optical coherence tomography, histopathology, ImageNet, Transfer, learning, impact, pretrained
News Source: Ophelia Keating. (November 7, 2026). Transfer Learning Effectiveness Varies Substantially Across Distinct Medical Imaging Modalities. Scienmag.
