Funnel ReLU outperforms standard activations in medical image deblurring benchmark

Choosing the right activation function may matter more for medical image restoration than most deep learning practitioners assume. A new empirical benchmark, published in Discover Artificial Intelligence, systematically compared fifteen activation functions across three medical imaging modalities and two widely used network architectures, and found that a spatially adaptive function called Funnel ReLU delivered the largest and most consistent gains over the industry-standard Rectified Linear Unit, particularly in deeper residual networks.

The study, conducted by Siju K. S., Vipin Venugopal, Mithun Kumar Kar and Malaya Kumar Nath, focused on medical image restoration: the task of recovering high-quality diagnostic images from degraded inputs such as blurred CT scans, compressed mammograms, or low-resolution MRI slices. The authors argue that activation function selection in this field has long been inherited from general computer vision practice rather than tested against the specific demands of clinical imaging, where fine details such as mammographic microcalcifications or lung tissue boundaries carry diagnostic weight.

To build a fair comparison, the team first surveyed the literature from 1943 to 2025, compiling a taxonomy of more than fifty activation functions drawn from 127 unique publications. From this pool they selected fifteen representatives spanning classical saturating functions like Sigmoid and Tanh, piecewise-linear rectifiers such as ReLU, Leaky ReLU and PReLU, exponential units like ELU, and modern smooth or spatially conditional functions including Swish, SiLU, GELU, Mish, Snake, GCU, ARiA and Funnel ReLU. The benchmark itself was conducted in three progressive phases: a fixed-weight analysis treating untrained networks as pure signal-processing filters, a short five-epoch screening of early learning dynamics, and a full-scale training evaluation.

The datasets comprised 1000 images from each of three grayscale modalities: axial lung CT slices, high-resolution mammography images with masses, and contrast-enhanced brain MRI slices. Because real clinical archives rarely contain matched low-quality and high-quality pairs, the researchers generated synthetic degradations. Deblurring inputs were produced by randomly applying one of three blur operators: Gaussian blur modeling optical point-spread, horizontal motion blur simulating patient movement, and disk-shaped defocus blur. Deblocking used JPEG compression at quality factors between 10 and 40, and a single-scale 2x super-resolution task was screened in the early phase. The authors are explicit that these synthetic operators are controlled abstractions that do not reproduce modality-specific degradation physics such as CT streak artifacts or MRI k-space ghosting.

Phase 2 screening showed that deblocking and super-resolution were relatively insensitive to activation choice, so the definitive Phase 3 benchmark concentrated on deblurring, training both a U-Net and an EDSR-style architecture for 100 epochs on 4000 training pairs per task. Performance was scored with five complementary metrics: Peak Signal-to-Noise Ratio, Structural Similarity Index, Edge Preservation Index, High-Frequency Energy Ratio, and the learned perceptual metric LPIPS. The authors caution that none of these metrics has been validated against radiologist scoring or lesion detectability; they serve only as proxies for low-level reconstruction quality.

The headline result concerns Funnel ReLU, a modern activation that differs from point-wise functions by including a trainable spatial condition built from a depthwise convolution, letting the activation adapt to surrounding texture and edges rather than processing each pixel in isolation. In the EDSR architecture, Funnel ReLU improved PSNR by 7.6 to 9.3 dB over the ReLU baseline across all three modalities, with Tukey HSD post-hoc tests confirming significance at p < 0.001 and Cohen’s d effect sizes of 3.2 to 4.4. The largest gains concentrated in the high-spatial-frequency band relevant to fine structures such as mammographic microcalcifications, which physically sit near the resolution limit of standard digital mammography systems.

Specific numbers illustrate the pattern. On CT deblurring, EDSR with ReLU reached 29.91 dB while EDSR with Funnel ReLU reached 37.48 dB, also surpassing the best U-Net configuration at 34.40 dB. On mammography, EDSR with Funnel ReLU achieved 46.49 dB and an SSIM of 0.990, against roughly 38.45 dB for EDSR with ReLU. On MRI, EDSR with Funnel ReLU hit 40.26 dB versus 30.97 dB for the ReLU baseline. Notably, the picture reversed in U-Net: there, ReLU was a strong baseline, and on mammography no candidate activation significantly improved PSNR over U-Net with ReLU, while some functions, including Mish, ELU and Sigmoid, produced statistically significant degradations. In U-Net on CT, SiLU delivered the most consistent statistical gains.

Because Funnel ReLU carries strictly more parameters than point-wise activations, the authors designed a matched-control experiment to rule out the simplest explanation, namely that its advantage merely reflects added capacity. They built ReLU_Spatial and GELU_Spatial controls with identical depthwise convolution and batch normalization branches, matched parameter counts and FLOPs, but combining the spatial branch by addition rather than Funnel ReLU’s max-gating. In EDSR on CT deblurring, these controls recovered only 3.8 to 8.1 percent of the gap between their base activations and Funnel ReLU, with differences not statistically distinguishable from zero, while Funnel ReLU exceeded both controls by about 7.3 dB. This supports the conclusion that the spatially adaptive max-gating mechanism itself, not the extra parameters, drives the benefit, though the authors note this validation was performed only for the CT deblurring task.

Statistical analysis reinforced the architecture dependence. One-way ANOVA showed a significant effect of activation choice, with p < 0.0001 across all combinations of architecture, imaging type and metric, and partial eta-squared values ranging from 0.12 to 0.72, mostly exceeding the conventional threshold for a large effect. Mechanistic analyses of feature maps and 2D FFT spectra showed that smooth functions like GELU and Mish produced bi-directional feature modulation, sigmoidal functions acted as strong low-pass filters, and Funnel ReLU showed spatially selective amplification with the broadest high-frequency spectral recovery. Sigmoid and Tanh consistently ranked worst, and the authors recommend avoiding them in hidden layers of deblurring networks.

The study carries important limitations that the authors state plainly. The literature survey was conducted by a single reviewer without a registered systematic protocol. Only three modalities and two architectures were tested, and the definitive evaluation covered deblurring only. All degradations were synthetic, with no validation on real clinical artifacts or expert radiologist assessment. Because patient-level metadata was unavailable, data splitting occurred at the 2D slice level rather than the patient level, meaning adjacent slices from the same subject could appear in both training and test subsets, potentially inflating absolute metrics; the authors therefore treat their numbers as indicative of relative restoration quality rather than fully generalizable estimates. Translating these findings into clinical benefit, they conclude, would require dedicated task-based reader studies on real clinical data.

Subject of Research: Empirical benchmarking of activation functions for deep learning-based medical image restoration

Article Title: An empirical benchmark of activation functions for medical image restoration networks

Article References: S, S. K., Venugopal, V., Kar, M. K., & Nath, M. K. (2026). An empirical benchmark of activation functions for medical image restoration networks. Discover Artificial Intelligence, 6(1), Article 1665. https://doi.org/10.1007/s44163-026-02265-1

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02265-1

Keywords: activation functions, medical image restoration, deep learning, Funnel ReLU, image deblurring, computed tomography, mammography, MRI, U-Net, EDSR, PSNR, high-frequency detail