In distribution networks where suppliers take responsibility for retailer inventory levels, calculating optimal restocking amounts while planning delivery paths forms an integrated inventory-routing problem that is computationally difficult to solve. Conventional industry workflows frequently depend on heuristic algorithms that approach inventory holding and delivery schedules separately or generate approximate solutions that degrade in quality as commercial networks expand.
Under the conventional vendor-managed inventory model, retailers transmit local inventory and demand data to a central vendor who oversees replenishment timing and product fulfillment. Although this operational arrangement shifts management burdens away from individual retail stores, executing both replenishment sizing and distribution planning concurrently presents severe algorithmic hurdles. When mathematical formulations merge inventory balancing with vehicle routing, the problem belongs to the class of NP-hard combinatorial optimization challenges. Traditional optimization routines commonly rely on genetic algorithms, particle swarm methods, simulated annealing, or mixed integer linear programming to search for viable schedules. However, these classical heuristic methods depend heavily on manual parameter adjustments, often get trapped in suboptimal solutions, and face substantial computational bottlenecks when applied to dynamic environments or scaled up across extensive supply networks.
To overcome these computational limitations, the authors constructed a Multi-Task Proximal Policy Optimization framework, designated as MTPPO, designed to resolve both inventory replenishment quantities and vehicle dispatching sequences within a unified training environment. The complete logistics problem is mathematically modeled as a Markov Decision Process running across a finite planning horizon. In this multi-task framework, the global state tracks current retailer inventory balances, previous replenishment quantities, observed historical demand, remaining vehicle load capacities, and existing routing links. Rather than relying on separate independent controllers, the architecture deploys an actor-critic configuration featuring two specialized actor neural networks paired with a single shared critic network. This shared critic continuously evaluates cumulative reward potential across the entire system, ensuring that routing maneuvers and inventory fulfillment choices are optimized in mutual cooperation rather than in isolation.
Handling the inventory replenishment sub-task requires determining continuous order quantities for each retail location while strictly avoiding stock-outs and remaining within designated storage limits. The inventory actor network adopts an encoder-decoder topology where initial retailer features—comprising geographical coordinates, current stock levels, past restocking volumes, and demand figures—are processed through a Graph Isomorphism Network. Graph Isomorphism Networks were specifically chosen because of their expressive capacity in extracting structured topological dependencies across retailer networks compared to alternative graph neural network architectures. The embedded spatial features are merged with historical state vectors and transferred through a multilayer perceptron decoder. This network calculates the mean and variance of a normal distribution from which continuous replenishment actions are drawn. The immediate reward function for the replenishment agent is structured as the negative sum of inventory holding costs and financial penalties associated with lost sales, thereby incentivizing high fulfillment rates.
Simultaneously, the routing sub-task determines the precise sequence in which a central supplier vehicle travels to deliver the assigned stock quantities to retail locations across the network. The routing state accounts for the completed distribution legs, the exact replenishment volumes calculated by the replenishment actor, the spatial coordinates of each retailer, and the current payload carried by the vehicle. The routing actor network processes the spatial coordinates and delivery requirements of all participating retail nodes through another Graph Isomorphism Network, feeding the resultant graph representations into an attention module. A softmax layer converts these attention scores into selection probabilities for candidate delivery nodes. The delivery vehicle departs from a single central depot, completes successive deliveries based on greedy probability selection, returns to the warehouse when empty or finished, and optimizes against a reward defined as the negative overall travel distance multiplied by unit delivery costs.
The joint training routine relies on Proximal Policy Optimization with a clipped surrogate objective function that balances updates across both continuous and discrete action spaces. By deriving a variance-reduced advantage estimator from the aggregated value function over the planning horizon, the critic guides parameter optimization for both actor networks simultaneously.
To evaluate the efficacy of the proposed model, the authors conducted computational experiments across four simulated retail scales consisting of 20, 50, 80, and 100 retailers over a 90-day operational evaluation period. The experimental environment was executed on an Ubuntu computational server equipped with an Intel Xeon Silver 4310 12-core processor running at 2.1 gigahertz, 256 gigabytes of system memory, and an NVIDIA Tesla A10 GPU with 24 gigabytes of memory. Demand across each retail location was treated as an uncertain random variable fluctuating between 0 and 50 units per time interval, while geographical coordinates were distributed randomly across a standardized unit coordinate plane. The performance of MTPPO was benchmarked against traditional genetic algorithm configurations—including independent inventory and routing models as well as integrated formulations—alongside deep reinforcement learning algorithms such as Deep Deterministic Policy Gradient for inventory replenishment and Asynchronous Advantage Actor-Critic for vehicle routing.
The quantitative results demonstrated a clear scaling divergence between heuristic algorithms and deep reinforcement learning architectures. However, as the retail environment expanded to 80 and 100 nodes, the learning capacity of the reinforcement learning framework became increasingly pronounced. In the 100-retailer scenario, MTPPO surpassed all benchmark combinations, achieving an 8.58 percent reduction in inventory costs and delivering a 6.18 percent decrease in total combined supply chain costs relative to the genetic algorithm benchmark. In separate shorter-term evaluations across a 30-day window, MTPPO maintained a 2.57 percent total cost advantage over the genetic algorithm.
In addition to total expenditure reductions, the multi-task reinforcement learning architecture demonstrated profound gains in operational execution speed, achieving an estimated 100-fold improvement in computational efficiency compared to the iterative search procedures required by heuristic baselines. Furthermore, the experiments established that the unified MTPPO framework incurred lower overall costs than combining decoupled single-task models, confirming that coordinating replenishment decisions directly alongside vehicle routing actions materially enhances the performance of both functional components.
Sensitivity analyses examining the transformer architecture within the routing module underscored the structural importance of multi-head attention mechanisms in handling complex distribution networks. Evaluating configurations with 1, 4, and 8 attention heads revealed that model performance improved as the number of heads increased, reaching minimum costs under an 8-head configuration. In the 100-retailer environment, expanding the attention mechanism from 1 to 4 heads reduced overall operational expenses by 4.68 percent, while advancing from 4 to 8 heads yielded an incremental cost reduction of 1.61 percent, demonstrating that high-dimensional feature transformations capture subtle spatial interactions up to a threshold of diminishing marginal returns.
Despite these demonstrable gains, the authors delineated clear operational boundaries and structural limitations inherent in their experimental formulation. The current mathematical model and reinforcement learning architecture assume a centralized logistics structure restricted to a single supplier depot and one serving vehicle executing sequential multi-trip distribution runs. Furthermore, delivery timing is determined exclusively by the central vendor without incorporating tight retailer-specified time windows, and distribution relies entirely on homogeneous conventional transport. The authors highlighted that extending future iterations of the multi-task framework will necessitate accommodating multiple supply warehouses, scaling routing actors to handle multi-vehicle fleets, incorporating explicit customer time-window constraints, and optimizing delivery schedules for heterogeneous vehicle fleets encompassing both electric and conventional transport options.
Subject of Research: Integrated inventory-routing optimization using multi-task deep reinforcement learning under vendor managed inventory
Article Title: Enhanced multi-task deep reinforcement learning for the integrated inventory-routing problem under VMI mode
Article References: Lu, G., Wan, J., Du, L., & Chen, X. (2025). Enhanced multi-task deep reinforcement learning for the integrated inventory-routing problem under VMI mode. Management System Engineering, 4(1), Article 17. https://doi.org/10.1007/s44176-025-00053-2
Image Credits: AI Generated
DOI: 10.1007/s44176-025-00053-2
Keywords: Vendor Managed Inventory, Inventory Routing Problem, Deep Reinforcement Learning, Proximal Policy Optimization, Multi-Task Learning, Graph Isomorphism Network, Supply Chain Optimization, Vehicle Routing, Attention Mechanism, Genetic Algorithm, Markov Decision Process, Actor Critic Architecture, Replenishment Planning, Logistics Engineering, Discrete Action Space, Continuous Action Space
News Source: Denise Maddox. (November 20, 2026). Multi-Task Deep Reinforcement Learning Cuts Supply Chain Inventory and Routing Costs. Scienmag.



