• HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
Thursday, November 19, 2026
BIOENGINEER.ORG
No Result
View All Result
  • Login
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
No Result
View All Result
Bioengineer.org
No Result
View All Result
Home NEWS Science News Technology

Simulated DDoS Detection Fails Cross-Environment Transfer Despite High Benchmark Accuracy

by
November 19, 2026
in Technology
Reading Time: 6 mins read
0
Simulated DDoS Detection Fails Cross-Environment Transfer Despite High Benchmark Accuracy

Simulated DDoS Detection Fails Cross-Environment Transfer Despite High Benchmark Accuracy

Share on FacebookShare on TwitterShare on LinkedinShare on RedditShare on Telegram

A diagnostic evaluation of machine-learning models trained to detect distributed denial-of-service attacks has revealed that high accuracy on a standard benchmark does not ensure successful detection across simulated network environments. Researchers trained eXtreme Gradient Boosting, Random Forest, Multilayer Perceptron, and majority-voting ensemble classifiers on the Canadian Institute for Cybersecurity Distributed Denial of Service 2019 benchmark dataset. When evaluated across ten frozen hold-out partitions using an eight-feature representation, the models achieved mean classification accuracies exceeding 98 percent. However, when these frozen models were applied without retraining to labeled network traffic generated inside the Objective Modular Network Testbed in C++ environment with the Internet Networking framework, detection performance dropped sharply depending on the specific attack scenario. Most notably, the consensus models failed to detect eight thousand simulated Transmission Control Protocol connection exhaustion attack flows.

Distributed denial-of-service attacks remain a persistent operational and financial challenge across computer networks because their distributed sources and varied traffic profiles complicate automated mitigation. While network defenders increasingly turn to artificial intelligence classifiers to identify malicious traffic patterns, acquiring representative, anomalous network traffic from real-world systems involves significant legal, ethical, and privacy constraints. Publicly available benchmark datasets provide accessible baselines, yet they often fail to capture the full breadth of novel attack configurations. To bypass real-world collection constraints, security researchers frequently deploy discrete-event network simulators such as OMNeT++ and INET to generate reproducible, labeled traffic within controlled network topologies. However, recent network research highlights that synthetic traffic cannot simply be assumed to match the underlying statistical distributions of established benchmark datasets without rigorous multi-metric verification.

To investigate the extent of cross-environment transfer, the research team established a two-stage experimental design. In the first phase, classifiers were trained and tested on the CICDDoS2019 dataset using a provenance-preserving data preparation pipeline. The original dataset inventory comprised eighteen comma-separated values files containing 113,828 raw benign records and over 70 million raw denial-of-service records. Data sanitization removed non-finite values and executed exact-record deduplication across normalized 88-column source rows, leaving 108,381 unique benign records. To avoid class imbalance and oversampling artifacts, exactly 108,381 attack records were sampled without replacement through a deterministic, day-, file-, and attack-stratified protocol, establishing a balanced development dataset of 216,762 total records. Ten frozen, source-group-aware partitions were constructed so that no source group, normalized record hash, or duplicate crossed between training and test sets.

The experimental framework examined nested feature sets of four, six, and eight variables designed to preserve consistent definitions across both benchmark captures and simulation pipelines. The base four-feature subset incorporated Total Forward Packets, Total Backward Packets, Flow Bytes per second, and Flow Packets per second. The six-feature configuration added Flow Duration and Total Length of Forward Packets, while the canonical eight-feature superset added Total Length of Backward Packets and Forward Packet Length Mean. Numeric features were scaled using Z-score standardization parameters derived exclusively from the training partitions to eliminate data leakage. The authors emphasized that feature reduction was implemented because these specific variables could be harmonized reliably across environments, rather than claiming the eight features represented an absolute globally optimal benchmark subset.

In the second experimental phase, researchers generated synthetic network traffic using OMNeT++ version 6.0.3 and INET version 4.5.4 across forty simulation runs representing ten independent random seeds across four scenarios. The canonical network topology incorporated fifty benign client nodes, twenty attacker nodes, ten Domain Name System reflectors, an access switch, an edge router, and a target server. Network links operated at 100 megabits per second with ten-meter cable distances, fifty-nanosecond propagation delays, and default interface queue limits of one thousand packets. Scenarios ran for 120 seconds with a ten-second initial warm-up period to eliminate transient effects, with attacks initiated between 20 and 21 seconds and terminated at 100 seconds. Across ten seeds per scenario, the run produced 10,312 uniquely labeled flows consisting of 2,012 benign flows and 8,300 attack flows.

The four simulation scenarios modeled distinct traffic behaviors: a normal baseline containing 500 benign flows of User Datagram Protocol echo packets; a UDP flood scenario with 512 benign flows and 200 attack flows of 1,024-byte packets; a DNS amplification scenario comprising 500 benign flows and 100 attack flows using 4,096-byte reflector responses; and a TCP connection exhaustion scenario containing 500 benign flows and 8,000 attack flows. The TCP scenario was implemented using twenty attacking hosts establishing parallel TCP session applications directed at port 80 with minimum one-byte payloads held open until 100 seconds. Bidirectional flow extraction enforced a 15-second idle timeout and a 120-second active duration limit. Flow ground truth was determined directly from frozen node and Internet Protocol identities rather than scenario labels or classifier outputs.

Benchmark evaluation on CICDDoS2019 demonstrated strong internal discrimination across all model families under the eight-feature specification. Mean hold-out classification accuracy reached 0.9861 for XGBoost, 0.9867 for Random Forest, 0.9836 for the Multilayer Perceptron, and 0.9862 for the majority-voting hybrid ensemble. Corresponding mean F1 scores were 0.9862, 0.9868, 0.9838, and 0.9864, while receiver operating characteristic area under the curve values averaged 0.9993 for XGBoost and Random Forest, 0.9974 for MLP, and 0.9991 for the hybrid ensemble. Matched statistical tests across split seeds showed that while the neural network was sensitive to feature set expansion from four to six and eight features, tree-based models maintained relatively stable, high-performance metrics across all evaluated representations.

When the frozen benchmark-trained classifiers were applied to the OMNeT++ traffic flows without retraining, the outcomes diverged markedly by scenario. Classification of normal benign traffic and UDP flood flows proved largely scenario-consistent for tree-based models and the hybrid ensemble. However, performance on DNS amplification varied across model architectures and feature subsets. Most critically, every principal eight-feature consensus model experienced a total transfer failure on TCP connection exhaustion, missing all 8,000 simulated attack flows. Consequently, across the aggregate 8,300 simulated attack flows in the entire experimental campaign, the eight-feature consensus models achieved an overall recall of only 3.6 percent, correctly identifying only 300 attack instances while generating 8,000 false negatives.

Analysis of classifier agreement further illustrated the nature of this collective breakdown. The majority-voting hybrid ensemble and XGBoost showed exceptionally high concordant behavior across all 10,312 simulated flows, with an average agreement of 0.9998, a Cohen’s kappa coefficient of 0.9962, and zero statistically significant paired McNemar differences across the ten test seeds after Holm correction. Random Forest similarly exhibited high agreement with XGBoost and the hybrid ensemble, whereas the Multilayer Perceptron demonstrated greater prediction variance and lower kappa values. The researchers highlighted that high inter-model agreement did not signify correct detection, as the tree ensembles and hybrid model consistently concurred on the exact same false-negative classifications during TCP connection exhaustion.

To investigate the features governing model behavior, the study conducted SHapley Additive exPlanations analysis using native TreeSHAP on the canonical eight-feature XGBoost model trained on seed 104,729, evaluating raw margins across 10,000 deterministically sampled hold-out observations. The team supplemented this with an auxiliary full-space analysis evaluating 79 eligible predictors derived from the 88-column CICDDoS2019 dataset. In the unrestricted 79-predictor model, variables such as minimum packet length, inbound direction, and initial TCP window sizes dominated TreeSHAP contributions. Among the eight harmonized variables, Flow Bytes per second ranked 18th, Flow Duration ranked 21st, and Flow Packets per second ranked 37th, while the remaining five cross-environment features received zero TreeSHAP attribution in that specific fit.

The authors clarified that zero attribution in an unrestricted model fit indicates a lack of utilization when dozens of correlated predictors are simultaneously available, rather than an absence of intrinsic predictive capability. However, the contrast between the features prioritized in unrestricted benchmark training and those available in compact cross-environment models highlights why transfer failures occur. Many high-ranking benchmark predictors, such as protocol flag counts and window sizes, lacked guaranteed semantic equivalence in the OMNeT++ extraction pipeline and were excluded from the cross-environment feature set. SHAP attributions provided mathematical diagnostic insight into feature weighting within the frozen classifiers, generating testable hypotheses regarding packet rates, durations, and byte volumes without establishing direct causal mechanisms within the simulator itself.

The study outlines several technical limitations governing the scope of these findings. The empirical results reflect the specific configurations of the CICDDoS2019 dataset, the evaluated OMNeT++ 6.0.3 and INET 4.5.4 parameters, and the tested network topology, precluding immediate generalization to all simulation platforms, traffic intensities, or real-world networks. Furthermore, the authors emphasized that machine-learning classification metrics and feature attribution methods do not constitute formal two-sample statistical tests of distributional fidelity, which would require dedicated divergence metrics such as Wasserstein distance or Maximum Mean Discrepancy. For network engineering and simulation communities, the researchers recommend developing built-in simulator export modules for detailed 80-to-100-feature sets, unifying metric naming standards, and calibrating packet generators with empirical burst templates rather than relying on high benchmark accuracy as a proxy for simulation realism.

Subject of Research: Evaluation of cross-environment transfer and explainability for machine learning DDoS detection in simulated networks

Article Title: AI-driven simulated DDoS attacks in OMNeT++ using SHAP-based explainability analysis

Article References: Amer, A., Shour, H., El Fawal, A. H., Nasser, A., Mansour, A., & Andrieux, G. (2026). AI-driven simulated DDoS attacks in OMNeT++ using SHAP-based explainability analysis. Discover Computing, 29(1), Article 910. https://doi.org/10.1007/s10791-026-10731-8

Image Credits: AI Generated

DOI: 10.1007/s10791-026-10731-8

Keywords: DDoS Detection, Network Simulation, OMNeT++, INET Framework, CICDDoS2019, Machine Learning, XGBoost, Random Forest, Multilayer Perceptron, Ensemble Learning, SHAP Analysis, Model Generalization, TCP Connection Exhaustion, UDP Flood, DNS Amplification, Intrusion Detection, Synthetic Network Traffic

News Source: Denise Maddox. (November 19, 2026). Simulated DDoS Detection Fails Cross-Environment Transfer Despite High Benchmark Accuracy. Scienmag.

Tags: CICDDoS2019DDoS detectionDNS Amplificationensemble learningINET Frameworkintrusion detectionMachine Learningmodel generalizationmultilayer perceptronnetwork simulationOMNeT++Random ForestSHAP analysisSynthetic Network TrafficTCP Connection ExhaustionUDP FloodXGBoost
Share12Tweet7Share2ShareShareShare1

Related Posts

Decoupled Genetic Algorithm Selects Compact Acoustic Features for Voice Pathology Detection

Decoupled Genetic Algorithm Selects Compact Acoustic Features for Voice Pathology Detection

November 19, 2026
Knowledge Transfer and Blockchain Capability in Saudi Supply Chain Financing

Knowledge Transfer and Blockchain Capability in Saudi Supply Chain Financing

November 19, 2026

Beryllium-Copper Flexure Spring Design Enhances Long-Life Stirling Cryocooler Compressor Reliability

November 19, 2026

Vision Guided Drone Landing Tests Compare Genetic Tuning and Q Learning

November 19, 2026

POPULAR NEWS

  • Engineered CAR Natural Killer Cells Target Mesothelin in Ovarian Cancer Models

    Engineered CAR Natural Killer Cells Target Mesothelin in Ovarian Cancer Models

    29 shares
    Share 12 Tweet 7
  • Industrial Winemaking Enzyme Cocktail Enables High Efficiency Genetic Transformation in Medicinal Mushrooms

    29 shares
    Share 12 Tweet 7
  • Infantile Case Report Links Novel NAXE Gene Variant to Fatal Encephalopathy

    29 shares
    Share 12 Tweet 7
  • Inherited Genetic Variants Significantly Shape Glioma Risk and Tumor Biology

    29 shares
    Share 12 Tweet 7

About

We bring you the latest biotechnology news from best research centers and universities around the world. Check our website.

Follow us

Recent News

Brief Implicit Association Test Shows Weak Ability to Distinguish Suicide Risk

Vision Transformer Model Identifies Eleven Small Mammal Species From Trackplate Footprints

Low-Dose Titanium Promotes Ginseng Growth and Saponin Biosynthesis in Controlled Pot Study

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 88 other subscribers
  • Contact Us

Bioengineer.org © Copyright 2023 All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Homepages
    • Home Page 1
    • Home Page 2
  • News
  • National
  • Business
  • Health
  • Lifestyle
  • Science

Bioengineer.org © Copyright 2023 All Rights Reserved.