Adversarial Attacks Distort Explainable AI Reasoning in Network and Malware Detection

The investigation evaluated three prominent explainability tools—DALEX for dataset-level global importance, SHAP for global and local attribution, and Integrated Gradients for local attribution—under the Fast Gradient Sign Method and multi-step Projected Gradient Descent. Rather than relying on isolated anecdotal examples or aggregate detection metrics, the researchers analyzed explanation stability over complete testing datasets. Their experimental framework separated instances where adversarial noise flipped the model’s classification decision from instances where the predicted label was fully preserved, revealing that explainability tools exhibit significant vulnerability independent of predictive failure.

The motivation for pairing explainability tooling with intrusion detection systems stems from the operational necessity to understand complex black-box machine learning decisions during active security incidents. Modern deep learning architectures achieve high detection accuracy on cyber threats, but security practitioners require transparent feature attributions to debug models, justify automated defensive actions, eliminate false alarms, and expose potential training biases. When an intrusion detection system issues an alert, analysts examine feature importance rankings to identify the specific indicators driving the classification, such as unusual protocol combinations, bursty network traffic, or suspicious executable memory allocations. However, if adversarial evasion techniques manipulate the internal attributions generated by explainability tools, malicious actors can reshape diagnostic outputs to highlight benign operational characteristics like common ports or routine system calls. Such distortions transform explainability mechanisms from verification assets into additional attack surfaces, potentially leading human defenders toward erroneous root causes and wasting critical incident response time.

To investigate this vulnerability, the researchers established an evaluation protocol that rigorously isolates explanation drift from prediction drift across two distinct cybersecurity benchmarks. Previous studies on adversarial explanation fragility focused predominantly on computer vision applications, where perceptual similarity constraints govern perturbations across continuous image pixels rather than engineered tabular metrics. Furthermore, prior cybersecurity implementations of explainable artificial intelligence typically evaluated feature attribution solely under clean, unperturbed conditions, measuring explanatory quality while overlooking stability under adversarial pressure. When earlier security literature did observe explanation movement under attack, it routinely conflated label changes with attribution changes, leaving unaddressed whether a security analyst monitoring model predictions alone could detect an underlying manipulation. By establishing full test set benchmarks and partitioning outcomes based on prediction retention, the current study directly addresses how explanations move when classifiers are attacked under a unified experimental schema.

The empirical evaluation utilized two diverse, domain-specific cybersecurity datasets characterized by fundamentally different feature semantics and class balances: CICMalMem22 and a refined variant of CICIDS17. CICMalMem22 serves as an obfuscated malware memory forensics benchmark comprising fifty-five numerical features extracted from memory dumps using the VolMemLyzer framework. These attributes represent system state counters categorized into five distinct functional groups: Malfind indicators of suspicious executable regions, Ldrmodule descriptors identifying manipulated modules, Handle statistics documenting referenced memory objects, Process-view metrics tracking operational relationships, and API-hook counts reflecting stealth mechanisms. The malware dataset maintains a balanced fifty-percent benign and fifty-percent malicious distribution, with malicious samples divided among Trojan Horse, Spyware, and Ransomware families. In contrast, the refined CICIDS17 benchmark captures network intrusion activity across seventy-two numeric attributes extracted via CICFlowMeter, eliminating categorical fields and erroneous instances. Its distribution is heavily skewed, containing eighty percent benign network flows and twenty percent attack traffic spanning eight distinct intrusion categories, including DDoS, Port Scan, and several DoS variants.

To establish baseline classification behaviors across both domains, the authors trained deep neural network models within Python 3.9 using the Keras 2.7 API and a TensorFlow computational backend. All numerical features underwent min-max normalization, linearly scaling inputs into the range of zero to one based exclusively on the training sets to prevent scale-induced feature dominance. Hyperparameter optimization was conducted via the Tree-structured Parzen Estimator algorithm using Hyperopt across thirty experimental trials, with a twenty-percent stratified validation split directing model selection. The resulting optimal neural network architecture incorporated two fully connected hidden layers with rectified linear unit activations, batch normalization, and dropout regularization, terminating in a softmax classification layer. Training proceeded for up to one hundred and fifty epochs with early stopping set to a patience of ten epochs, successfully restoring model checkpoints with the lowest validation loss while avoiding diminishing performance returns.

Baseline predictive evaluation confirmed that while global detection metrics remained robust, performance varied substantially across individual threat categories on both datasets. On CICIDS17, the neural network achieved higher aggregate performance metrics, driven by dominant classes such as Normal flows with an F1-score of 0.96, DoS Hulk with 0.97, and DDoS with 0.99. However, minority categories proved far more challenging, with DoS slowloris and DoS GoldenEye recording F1-scores of only 0.21 and 0.36, respectively. On CICMalMem22, the model achieved a perfect 1.00 F1-score for benign instances, but classification performance dropped to 0.58 for Trojan samples, 0.55 for Spyware, and 0.51 for Ransomware. Despite these baseline class disparities, the macro-averaged F1-scores between the two benchmark datasets remained comparable, providing a reliable baseline classifier for subsequent adversarial testing.

The threat model posited a white-box adversary operating at inference time with complete knowledge of the trained neural network parameters and gradient information. Constrained within an infinity-norm perturbation ball of radius epsilon, the attacker executed untargeted evasion attacks aimed at maximizing the categorical cross-entropy loss of the true class. Two gradient-based attacks were evaluated: the single-step Fast Gradient Sign Method and the multi-step Projected Gradient Descent algorithm, using a baseline perturbation factor of epsilon equal to 0.01. To measure the resulting explanation movement across clean and perturbed inputs, the authors computed three complementary quantitative metrics across the complete test splits: the Top-k feature overlap for the top five and top ten attributes, the cosine similarity between attribution vectors, and the L1 attribution distance. For all local explainers, attribution vectors on both clean and attacked samples were generated strictly with respect to the clean predicted class, ensuring that measured shifts reflected actual attribution movement rather than comparisons across disparate output neurons.

Crucially, the experimental results revealed that substantial explanation drift occurs even when adversarial inputs fail to alter the classifier’s final categorical prediction. Across the evaluated configurations, between 48.0 percent and 88.4 percent of the classifier predictions were successfully retained under attack. Within this prediction-preserved partition, the mean Top-5 feature overlap ranged from 0.586 to 0.892, while cosine similarity values ranged between 0.875 and 0.962. This quantitative finding demonstrates that between 10.8 percent and 41.4 percent of the five most influential features shifted their positions in the explanatory narrative without triggering any detectable change in the predicted label. The most severe preserved-prediction drift was observed for SHAP on the CICMalMem22 malware dataset under untargeted Projected Gradient Descent, where the mean Top-5 feature overlap dropped to 0.586. While label flips further amplified explanatory divergence, they did not account for the entirety of attribution displacement, proving that prediction-preserving drift constitutes a tangible operational vulnerability.

Comparative analysis among the explainability frameworks indicated that Integrated Gradients consistently maintained higher attribution stability than SHAP across both benchmark datasets. Integrated Gradients achieved superior Top-k feature overlap and higher cosine similarity scores, whereas SHAP demonstrated greater vulnerability to adversarial noise, particularly within the memory forensic domain. Meanwhile, global explanations generated through DALEX exhibited comparatively strong overall stability, maintaining full-ranking Spearman rank correlations between 0.848 and 0.900 across both datasets. Nevertheless, DALEX rankings showed notable fragility at the very top of their feature hierarchies on CICMalMem22, where only three of the five most critical features survived under both attack types. On CICIDS17, DALEX proved more resilient, preserving four of five top features under the Fast Gradient Sign Method and all five under Projected Gradient Descent. The experimental data also demonstrated that attack severity does not follow a universal hierarchy: while Projected Gradient Descent induced greater drift for SHAP on both benchmarks, Integrated Gradients was more stable under Projected Gradient Descent on CICMalMem22.

A sensitivity analysis evaluating Integrated Gradients under Projected Gradient Descent across varying perturbation factors demonstrated that explanation stability degrades monotonically as perturbation strength increases. Testing perturbation values across 0.005, 0.01, 0.05, and 0.1 revealed steep reductions in both Top-5 feature overlap and cosine similarity alongside catastrophic declines in prediction retention. When the perturbation budget expanded to 0.05, prediction retention plummeted to 19.11 percent on CICIDS17 and just 0.05 percent on CICMalMem22, deteriorating further at 0.1 to 0.47 percent and 0.01 percent, respectively. These measurements confirmed that the primary baseline setting of 0.01 provided an optimal threshold for isolating explanation drift, retaining 87.98 percent of predictions on CICIDS17 and 47.97 percent on CICMalMem22, thereby enabling robust statistical analysis on substantial prediction-preserved sample subsets.

The researchers noted essential scope boundaries, emphasizing that their findings apply specifically to gradient-based evasion attacks on neural networks utilizing tabular cybersecurity features. Because the adversarial generation process optimized classifier loss rather than an explicit explanation-manipulation objective, the observed drift represents an incidental consequence of conventional evasion techniques rather than a worst-case scenario. Additionally, the perturbations respected mathematical range bounds without enforcing domain-specific semantic dependencies among interrelated network traffic or memory state counters. The authors also highlighted that SHAP estimations derived from KernelExplainer incorporate stochastic sampling variance, whereas Integrated Gradients relies on deterministic gradient integration, suggesting that part of SHAP’s higher measured drift may reflect sampling variance. Concluding their assessment, the authors emphasized that future security validations must evaluate explanation-robustness metrics alongside traditional accuracy and F1-scores, warning that defensive strategies relying on attribution signatures to detect adversarial inputs cannot presuppose explanation stability under attack.

Subject of Research: Evaluating the robustness of explainable artificial intelligence methods under adversarial attacks in intrusion and malware detection

Article Title: Evaluating the robustness of explainable AI methods to adversarial attacks in intrusion and malware detection

Article References: AL-Essa, M., Atoum, M., Imran, M., ALi, W. A., Alassaf, N., & Alsaadi, M. S. (2026). Evaluating the robustness of explainable AI methods to adversarial attacks in intrusion and malware detection. Discover Artificial Intelligence, 6(1), Article 1811. https://doi.org/10.1007/s44163-026-02587-0

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02587-0

Keywords: explainable AI, adversarial attacks, intrusion detection systems, malware detection, SHAP, Integrated Gradients, DALEX, cybersecurity, machine learning robustness, Evaluating, robustness, explainable