Deep Learning in Medicine Hits a Wall: Explainability and Privacy Are Locked in Tension

Deep learning has transformed the way machines read the human body. Convolutional neural networks now flag tumors on MRI scans, detect COVID-19 from chest X-rays, and even authenticate individuals from the electrical signature of their heartbeat. Yet a sweeping new review argues that the technology’s clinical future hinges on solving a problem that has been hiding in plain sight: the very tools designed to make these systems trustworthy — explainability and privacy — are quietly working against each other. In a study published in Multimedia Tools and Applications, Ghazala Hcini and Imen Jdey of the ReGIM-Lab at the National Engineering School of Sfax, Tunisia, with Jdey also affiliated with CITI lab at INSA de Lyon in France, present one of the first systematic attempts to examine these two imperatives together rather than in isolation.

The researchers analyzed 61 studies published between 2020 and 2026, all of which explored interpretable or privacy-preserving deep learning techniques in medical imaging and biometric contexts. Their comparison of prior surveys revealed a striking gap in the literature: no previous review had simultaneously addressed explainability, privacy, medical data, and biometric data. Most surveys, they found, tackle one dimension at a time, often within a narrow architectural or clinical niche. That fragmentation matters, the authors argue, because in real healthcare deployments the two requirements collide — and the collision has consequences for patient safety, regulatory compliance, and public trust in artificial intelligence.

The core of the problem is what the review calls a fundamental tension. Explainability techniques such as saliency maps, gradient-based attribution methods like Grad-CAM and integrated gradients, and local surrogate models like LIME and SHAP all work by interrogating a model’s internal representations. They reveal which pixels, features, or signal components drove a particular prediction. But those same internal representations can encode private information. A model trained to diagnose pneumonia from chest X-rays may also learn features that allow it to re-identify the patient, a phenomenon demonstrated in prior research showing that deep learning can exploit the biometric nature of medical imaging data. In other words, an explanation that faithfully exposes what the model learned may inadvertently expose who the patient is.

The reverse direction of the trade-off is equally troubling. Privacy-preserving techniques — federated learning, differential privacy, k-anonymity, homomorphic encryption, and learnable image encryption — deliberately obscure or perturb data and model updates to protect sensitive information. Differential privacy, for instance, calibrates statistical noise to individual data sensitivity, while federated learning keeps raw images on hospital servers and shares only model parameters. But the noise and abstraction that shield patients can also mask the very representations that attribution methods depend on. Recent work cited in the review has shown that differential privacy and federated learning can measurably distort the outputs of explanation algorithms, producing attributions that are less faithful to the model’s actual reasoning. A clinician looking at a heatmap under these conditions may be reading a blurred, unreliable account of why the algorithm made its call.

The stakes of this tension are not abstract. The review documents how explainable AI has been applied across an extraordinary range of clinical tasks: LIME-based interpretation of brain tumor detection on MRI, Grad-CAM visualization for skin cancer diagnosis with vision transformers, attention-transfer networks for COVID-19 radiographic diagnosis, and layer-wise relevance propagation for Alzheimer’s classification. In the biometric domain, deep learning systems authenticate users from electrocardiogram signals, retinal vessel patterns, and electroencephalographic activity. Each application demands both a defensible explanation for its decisions and rigorous protection of data that is, by definition, deeply personal. An ECG recording is simultaneously a diagnostic signal and a biometric identifier; a retinal image can screen for disease and unlock a phone.

Compounding the challenge, the review catalogs a growing arsenal of attacks that exploit the intersection of these domains. Membership inference attacks can determine whether a specific individual’s record was used in training. Model inversion attacks can reconstruct training images from model parameters. Adversarial methods can even manipulate explanation techniques themselves, crafting models whose saliency maps look plausible while the underlying decision logic is compromised. The authors point to surveys documenting these privacy attacks and defenses, noting that explanations expand the attack surface: by exposing more about a model’s internals, they give adversaries more material to work with. Trustworthy systems, they argue, must be designed with this adversarial reality in mind from the outset.

Federated learning emerges in the review as the most heavily researched bridge between the two imperatives. Studies covered include federated frameworks for COVID-19 detection from chest X-rays, brain tumor classification with integrated explainable AI, privacy-preserving malaria image detection, and explainable federated models for skin cancer diagnosis and eye disease detection. Multinational validation studies have shown that federated deep learning can match centralized performance for detecting lung abnormalities in CT scans without moving patient data across borders. Yet the authors caution that federated learning alone is not a panacea: shared model updates can still leak information, aggregation strategies remain an active research area, and the interaction between federated training and explanation quality is only beginning to be understood systematically.

Beyond the technical layer, the review situates the problem within a tightening regulatory landscape. The European Union’s GDPR established a de facto “right to explanation” for algorithmic decisions, while health-specific frameworks such as HIPAA in the United States govern the handling of protected health information. Emerging regulations on artificial intelligence add further obligations around transparency and accountability. The authors note that compliance with these frameworks cannot be achieved by treating explainability and privacy as separate checkboxes; a system that satisfies one while undermining the other may fail both in practice. Their synthesis positions the joint adoption of interpretable and privacy-preserving methods as a foundational necessity for next-generation medical and biometric imaging, not an optional enhancement.

What would genuinely trustworthy clinical AI look like? The review sketches the contours of an answer. It calls for evaluation protocols that measure explanation faithfulness under privacy constraints, rather than assessing each property in isolation. It highlights promising directions such as privacy-preserving generative adversarial networks that produce case-based explanations without revealing real patient images, explainable privacy-preserving image compression, and frameworks that quantify the privacy risks of model explanations themselves. It also echoes a broader debate in the machine learning community about whether high-stakes decisions should rely on inherently interpretable models rather than post-hoc explanations of black boxes. The authors’ contribution is to insist that these questions be asked simultaneously, across both medical and biometric data, and with the adversarial landscape fully in view.

The message for clinicians, developers, and regulators is ultimately one of cautious urgency. Deep learning’s diagnostic prowess is no longer in doubt; what remains uncertain is whether the systems built on it can earn the trust of the people who must use them and the patients they affect. By mapping 61 studies onto a single coherent framework, Hcini and Jdey have made the trade-off visible in a way scattered individual papers could not. Resolving it, they conclude, is not merely advantageous but necessary — and the field now has a clearer map of where the conflicts lie, which methods show promise, and which open problems will define the next generation of trustworthy medical and biometric AI.

Subject of Research: Explainability and privacy trade-offs in deep learning for medical imaging and biometric data analysis

Article Title: Trustworthy analysis of medical and biometric data with deep learning: Methods, challenges, and the imperatives of explainability and privacy

Article References: Hcini, G., & Jdey, I. (2026). Trustworthy analysis of medical and biometric data with deep learning: Methods, challenges, and the imperatives of explainability and privacy. Multimedia Tools and Applications, 85(9), Article 744. https://doi.org/10.1007/s11042-026-21841-2

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21841-2

Keywords: deep learning, explainable AI, privacy, medical imaging, biometrics, federated learning, differential privacy, Grad-CAM, LIME, health informatics, data protection, clinical AI