Tree Models Explain Intrusion Detection More Faithfully Than Deep Learning

Tree-based machine learning models provide more faithful explanations for network intrusion detection than deep learning architectures when evaluated on live adversarial traffic, according to a study published in Frontiers in Computer Science. Researchers from REVA University in Bengaluru, India, deployed a multi-service honeypot system to collect real-world attack data and tested whether explainable artificial intelligence (XAI) methods accurately reflect the features driving model predictions. The study represents one of the first evaluations of explanation faithfulness on operational telemetry rather than curated benchmark datasets, addressing a critical gap in the reliability of AI-driven security tools.

The research team addressed three primary questions: whether explanation attributions remain faithful on live adversarial traffic, if cross-method attribution consistency holds under operational conditions, and whether tree-ensemble explanations remain more faithful than deep-model explanations on live data. To answer these, the authors deployed ten honeypot services on Google Cloud Platform for 21 days, capturing 10,900 sessions from 4,817 distinct IP addresses across 73 countries. This operational approach allowed the evaluation of models in a setting that mimics the noisy, unstructured environment of a security operations center, where analysts face alert fatigue and opaque AI decisions.

The dataset was labeled through a human-in-the-loop process involving two domain experts with over five years of security operations experience. Each session was classified as either malicious or benign-noise, with inter-rater reliability quantified at a Cohen’s kappa of 0.91. The final dataset comprised 5,486 malicious sessions and 5,414 benign-noise sessions, creating a near-balanced class ratio of 1.01:1. This labeling process included automated pre-labeling, entropy-based anomaly scoring, and expert consensus review to mitigate circularity between labels and features, although the authors note that a complete feature-blind re-labeling study remains a limitation.

Feature engineering produced a 13-dimensional vector for each session, incorporating protocol-specific behavioral, statistical, and temporal signatures. Key features included payload length, command frequency, hour of day, unique commands, and authentication failure ratio. The team trained six distinct model architectures: Random Forest, XGBoost, LightGBM, Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM), and Transformer. All models were evaluated using 10-fold stratified cross-validation, with a separate 15% holdout set reserved for frozen checkpoint evaluation to ensure no data leakage during the faithfulness assessment.

Model performance on the holdout set showed that Random Forest achieved the highest F1 score of 88.6%, followed by XGBoost at 88.2% and LightGBM at 88.1%. The deep learning models performed slightly lower, with the MLP at 87.7% and the LSTM at 83.4%. The Transformer model failed to converge, predicting every session as malicious and yielding a constant F1 score of 67.0% and an AUC of 50.0%, effectively rendering it useless for this tabular classification task. This convergence failure served as a sanity check, demonstrating that the faithfulness metrics could distinguish between models with valid decision boundaries and those that ignored input features entirely.

To evaluate explanation faithfulness, the researchers used two complementary metrics: the Area Under the Descriptive-Accuracy Curve (AUDAC) and the perturbation flip rate. AUDAC measures how quickly a model’s accuracy drops when its top-ranked features are sequentially masked; a lower AUDAC indicates that the model relies more strongly on the features identified by the explanation method. The tree ensembles exhibited lower AUDAC values than the deep models, with LightGBM at 0.346, XGBoost at 0.348, and Random Forest at 0.351. In contrast, the MLP had an AUDAC of 0.442 and the LSTM 0.458, suggesting that their explanations were less faithful to the actual decision process.

The perturbation flip rate analysis further supported the superiority of tree-based models. When the top 10 SHAP-ranked features were masked with median values, 100% of correctly classified malicious sessions changed their prediction for Random Forest, XGBoost, and LightGBM. The MLP flipped 98.8% of malicious predictions, while the LSTM flipped 87.0%. Under random masking, only about 61% of malicious sessions flipped, highlighting that the SHAP-ranked features were indeed driving the predictions. Benign predictions were largely stable, with tree ensembles showing 0% benign flips at seed 42, and the MLP showing only 2.2%.

Cross-method consistency between SHAP and LIME explanations was strong, with Spearman rank correlations ranging from 0.85 to 0.93 across the five converged models. This high global correlation suggests that analysts would see similar feature rankings regardless of which explanation method or model family was used, facilitating trust in the system. However, instance-level agreement was lower, with median correlations between 0.43 and 0.72, indicating that local explanations for specific sessions can vary significantly between methods. This distinction is crucial for SOC workflows, where global trends inform policy but local explanations guide individual incident response.

Ablation studies revealed that the models were not simply replicating signature-based rules. Removing the top three behavioral features used by honeypot signatures resulted in only small reductions in F1 scores, ranging from 3.3 to 3.8 percentage points. This suggests that the models were leveraging broader contextual information, such as payload entropy and timing patterns, rather than relying on obvious attack indicators. Additionally, leave-one-service-out analysis showed no evidence of service-identity shortcuts, with mean F1 changes under 0.6 percentage points for all models, indicating robust generalization across different service types.

The study highlights several limitations, including the short 21-day collection window, which may not capture seasonal attack variations, and the single-region deployment, which limits geographic generalizability. The binary classification framework does not distinguish between attack subcategories, and the use of library-default hyperparameters for deep models may underestimate their potential performance. Furthermore, the Transformer’s failure to converge on this tabular dataset underscores the challenge of applying sequential architectures to non-sequential data. Despite these constraints, the study provides a reproducible pipeline for evaluating XAI reliability on live data, offering a design rationale for deploying explanation methods in operational security environments.

The findings suggest that tree-based models are currently more reliable for explainable intrusion detection on tabular session-level features. While deep learning models offer theoretical advantages for complex temporal data, their explanations in this context were less faithful and more sensitive to initialization seeds. For security operations centers, this implies that adopting tree ensembles with SHAP and LIME explanations may provide a more transparent and stable foundation for analyst decision-making. Future work should explore multi-class attack taxonomies and formal user studies to validate the practical impact of these explanations on analyst performance and trust.

Beyond the headline metrics, the honeypot architecture revealed operational nuances across its deployed decoys and responders. The sensor framework incorporated ten containerized services alongside three single-process host banner responders covering File Transfer Protocol, Domain Name System, and Virtual Network Computing, plus a catch-all listener for remaining ports. Of the seventeen listening destination ports, five generated zero sessions throughout the three-week collection period: Server Message Block on port 445, Telnet on port 23, Modbus on port 502, Jupyter on port 8888, and the Kubernetes application programming interface on port 6443. The authors noted that the total absence of traffic on commonly targeted ports like 445 and 23 most plausibly indicated upstream network filtering by the cloud infrastructure rather than a genuine absence of attacker scanning. The remaining twelve qualifying port categories received between 651 and 744 sessions each following per-service capping, accounting for 1,301 of the 1,635 held-out records, while the host-level catch-all responder logged 334 sessions across 2,297 non-instrumented destination ports.

The investigators also subjected the classifiers to rigorous non-random partitioning protocols to gauge resilience against distribution shifts. In an hour-of-day temporal holdout where models trained on sessions initiating between 00:00 and 17:00 UTC and were tested on hours 18:00 through 23:00 UTC, the panel mean F1 score dropped by 19.3 percentage points to 68.6 percent. This sharp decline reflected a substantial class distribution shift, with malicious traffic dropping from 61.2 percent during training hours to 32.1 percent during evening testing hours. Despite the performance drop, the architectural ranking remained identical to cross-validation results, led by Random Forest at 73.9 percent. In a separate source-address grouping check where all sessions from any single internet protocol address were quarantined on one side of the split, the panel mean F1 score registered 79.2 percent, an 8.7 percentage point decrease compared to standard cross-validation, demonstrating moderate degradation when models cannot exploit repeated interactions from identical remote sources.

Sensitivity testing across three random seeds (42, 7, and 2024) exposed marked differences in stability among the tree architectures that single-run evaluations obscured. While Random Forest maintained consistent descriptive accuracy with an area under the curve of 0.341 and a standard deviation of 0.018 alongside an unwavering 100 percent malicious flip rate, XGBoost demonstrated considerable sensitivity to split variations. Across the three seeds, XGBoost’s descriptive-accuracy metric averaged 0.428 with a standard deviation of 0.086, and its malicious perturbation flip rate ranged from 71.8 percent to 100.0 percent with benign flips reaching an average of 8.3 percent. Consequently, XGBoost’s explanation fidelity occasionally ranked below the multilayer perceptron, tempering conclusions regarding uniform tree-ensemble superiority. Finally, runtime telemetry confirmed practical feasibility for operational deployment: LightGBM proved fastest with a single-event inference latency of 10.8 milliseconds and a throughput of 926 events per second, closely trailed by Random Forest at 12.4 milliseconds and XGBoost at 13.1 milliseconds, whereas the recurrent neural network required 43.2 milliseconds.

Subject of Research: Evaluation of explainable AI faithfulness in live honeypot intrusion detection

Article Title: Are explanations faithful under fire? Evaluating XAI reliability on live honeypot intrusion-detection telemetry

Article References: Sridhar, G., Simha, J. B., & Agarwal, R. (2026). Are explanations faithful under fire? Evaluating XAI reliability on live honeypot intrusion-detection telemetry. Frontiers in Computer Science, 8, Article 1972293. https://doi.org/10.3389/fcomp.2026.1972293

Image Credits: AI Generated

DOI: 10.3389/fcomp.2026.1972293

Keywords: Explainable AI, Honeypot, Intrusion Detection, Random Forest, XGBoost, LightGBM, SHAP, LIME, Cybersecurity, Machine Learning, Deep Learning, Transformer, LSTM, AUDAC, Flip Rate, SOC, Telemetry, Feature Importance, Model Faithfulness, Adversarial Traffic