The study, published in Discover Artificial Intelligence, introduces a model called Prioritized Experience Replay Dueling Double Deep Q-Network, or PER-Dueling Double-DQN. The primary objective of this work is to address the challenges posed by dynamic workloads and heterogeneous resources in distributed systems.
These include a Dueling network architecture, Prioritized Experience Replay, and Double-DQN learning. The Dueling architecture allows the model to separately estimate the value of a system state and the advantage of specific scheduling actions. Double-DQN is used to reduce the overestimation of Q-values, a common issue in standard Deep Q-Networks that can lead to unstable convergence. Prioritized Experience Replay improves sample efficiency by assigning higher sampling probabilities to transitions with larger temporal-difference errors, ensuring that the most informative experiences are replayed more frequently during training.
The system was evaluated using a trace-driven simulation based on the Alibaba Cluster Trace v2017 dataset. This dataset contains production traces collected from approximately 1,300 machines over a 12-hour period, including both long-running online services and batch workloads. After preprocessing, which involved removing duplicates and handling missing records, the researchers retained 17,414 valid tasks for the experimental evaluation. The simulated testbed incorporated a heterogeneous architecture consisting of 25 Fog servers and 25 Cloud virtual machines. The Fog nodes were modeled with moderate processing capabilities and low communication latency, while the Cloud resources were modeled with higher computational capacity but increased transmission delays.
The scheduling problem was formulated as a sequential decision-making process where the agent must assign each incoming task to exactly one execution node. The state representation provided to the agent included task characteristics such as CPU demand, memory requirement, execution duration, remaining deadline, and Quality of Service priority. This task information was combined with real-time resource information for all Fog and Cloud nodes, including available CPU, available memory, current processing load, resource availability time, and energy coefficients.
A multi-objective reward function was designed to guide the agent’s learning process. This function jointly considered latency, energy consumption, deadline compliance, Quality of Service requirements, resource overload, and the selected execution layer. The latency component was defined to reward lower execution times, while the energy component penalized higher consumption. Energy costs were expressed in simulation units rather than physical units such as Joules, with specific coefficients set for Fog and Cloud resources to reflect their relative energy costs. Deadline violations were penalized with a fixed negative reward, and resource overload was penalized based on the maximum normalized CPU or memory deficit. A small execution-layer term was included to moderately favor low-latency Fog execution without completely excluding Cloud resources.
The experimental results indicated that the proposed PER-Dueling Double-DQN framework achieved lower mean latency, energy consumption, and SLA violation rates compared to several baseline algorithms. The baselines included Double-DQN, DQN, DDPG, LSTM-RNN, First-Come-First-Served, and Earliest Deadline First. The evaluation was conducted over 10 independent runs using random seeds from 42 to 51 to ensure statistical robustness. The proposed method demonstrated the lowest average task latency among all compared algorithms, suggesting that the integration of Dueling and Prioritized Experience Replay mechanisms effectively improves the agent’s ability to adapt to time-varying workloads and heterogeneous resource conditions.
Statistical significance analyses were conducted to validate the performance differences observed between the proposed method and the learning-based baselines. A Friedman test revealed statistically significant differences among the five learning-based scheduling methods for all evaluated performance metrics, including cumulative reward, SLA violation rate, latency, and energy consumption. Post-hoc pairwise comparisons using the Wilcoxon signed-rank test with Holm correction showed that the proposed method exhibited statistically significant differences in cumulative reward and SLA violation rate compared with all evaluated learning-based baselines. However, the differences in latency compared to Double-DQN and in energy consumption compared to Double-DQN and DQN were not statistically significant, indicating that while the proposed method performs well, some improvements over specific baselines are marginal in the context of the simulation settings.
An ablation study was performed to quantify the individual contributions of the Dueling architecture and Prioritized Experience Replay to the overall performance of the framework. The study compared the complete PER-Dueling Double-DQN framework with variants that excluded specific components, such as DDQN, Dueling-DDQN, and PER-DDQN. The results showed that the complete framework achieved the lowest reported mean latency, energy consumption, and SLA violation rate, along with the highest mean cumulative reward. This suggests that the combination of these learning components is beneficial for task scheduling in the evaluated experimental configuration, as each component addresses different limitations of standard Deep Q-Networks, such as Q-value overestimation and inefficient sample utilization.
Scalability analysis was also conducted to investigate the performance of the scheduler under different infrastructure sizes. Experiments were run with configurations of 10 Fog and 10 Cloud nodes, 25 Fog and 25 Cloud nodes, and 50 Fog and 50 Cloud nodes. The results showed that the scheduler maintained stable performance as the infrastructure increased from 20 to 100 execution nodes. The average task latency remained approximately 14.59 ms across these configurations, while the SLA violation rate decreased as additional execution resources became available. The computational overhead analysis indicated that the per-task inference time remained below approximately 0.61 ms, suggesting that the online decision overhead remains low even as the action space expands significantly.
The study highlights the potential of deep reinforcement learning for managing complex scheduling tasks in IoT-Fog-Cloud environments. By leveraging real-world workload traces, the researchers were able to evaluate the framework under conditions that reflect production environments, although the infrastructure itself was simulated. The findings indicate that the proposed approach can effectively balance conflicting objectives such as latency, energy consumption, and SLA compliance. However, the authors note that the results are limited to the considered trace-driven simulation environment and do not represent a physical IoT-Fog-Cloud deployment. Future work will explore multi-agent reinforcement learning for cooperative scheduling among distributed Fog nodes and the integration of dynamic mechanisms for task migration and preemption to further improve SLA compliance under bursty workloads.
Subject of Research: Prioritized replay dueling double DQN for intelligent task scheduling in IoT Fog Cloud computing
Article Title: Prioritized replay dueling double DQN for intelligent task scheduling in IoT Fog Cloud computing
Article References: Jawhar, N., Talha, A., & Bouayad, A. (2026). Prioritized replay dueling double DQN for intelligent task scheduling in IoT Fog Cloud computing. Discover Artificial Intelligence, 6(1), Article 1711. https://doi.org/10.1007/s44163-026-02614-0
Image Credits: AI Generated
DOI: 10.1007/s44163-026-02614-0
Keywords: Prioritized, replay, dueling, double, intelligent, task, scheduling, Cloud, computing, scientific research
News Source: Denise Maddox. (November 7, 2026). AI Scheduling Framework Reduces Latency in Simulated IoT Fog Clouds. Scienmag.
