Reinforcement learning has always had a quiet flaw at its heart: most of the algorithms that power it assume the world stays still. Now a team of German researchers has shown that one of the most promising quantum-enhanced learning machines not only survives a shifting world but can outpace its classical rivals in adapting to it — a result that pushes quantum machine learning a step closer to the messy, ever-changing environments of real applications.
In a study published in the journal Quantum Machine Intelligence, Oliver Sefrin of the German Aerospace Center (DLR) in Ulm, together with Manuel Radons and Lars Simon of Bundesdruckerei GmbH in Berlin and Sabine Wölk of DLR and the Center for Integrated Quantum Science and Technology at Ulm University, report the first systematic investigation of a hybrid quantum-classical reinforcement learning agent in non-stationary environments. The work, published as an open-access research article in Volume 8 of the journal, marks a departure from earlier studies of the so-called hybrid agent for quantum-accessible reinforcement learning, which had until now been tested only in stationary settings where the rules of the game never change.
Reinforcement learning, or RL, is the branch of machine learning in which an agent learns by trial and error, interacting with an environment that is formally described as a Markov decision process. The agent observes states, takes actions, collects rewards, and gradually adjusts its policy — the probability with which it chooses each action in each state — to maximize cumulative reward. From mastering Atari games and Go to fine-tuning today’s large language models through reinforcement learning from human feedback, the paradigm has become one of the most consequential tools in artificial intelligence. But in realistic scenarios, from traffic management to industrial control, the environment itself moves: lanes close, demand fluctuates, targets relocate. A policy that was optimal yesterday may be worthless today, and well-trained agents often fare worse after a sudden change than untrained ones, because what they have learned actively works against them.
The hybrid learning agent at the center of the new study, introduced in earlier work by Hamann and Wölk, offers a provable speed advantage in a special class of learning problems. It combines a classical RL algorithm with Grover-type amplitude amplification, a quantum technique that boosts the probability of sampling rewarded action sequences quadratically faster than random classical sampling. In the quantum part of the algorithm, actions and percepts are encoded in orthonormal quantum states, and the environment’s response is modeled as a unitary that produces a phase-kickback oracle marking rewarded action sequences with a sign flip. Applying Grover iterations to a superposition of action sequences weighted by the agent’s current policy raises the chance of measuring a rewarded sequence according to the well-known sine-squared amplification law, and the resulting candidate is then verified in a classical episode of play. The payoff is a quadratic reduction in the number of episodes needed to find rewarded behavior — but only for deterministic, strictly episodic tasks, such as the classic Gridworld maze, where the episode length is fixed and transitions are deterministic.
The catch, the researchers recognized, is that speed can become a liability. “In such dynamic learning scenarios, a well-trained agent often performs worse after a sudden change of the environment compared to an untrained or a barely trained agent,” the authors note in the paper. A fast-learning quantum agent that has converged hard onto one reward structure can find itself trapped, its confident policy pointing away from the new source of reward. To be competitive outside the laboratory, the hybrid agent needed a way to forget.
The team’s solution is elegantly simple: they paired the hybrid agent with Projective Simulation, a tabular RL algorithm developed by Hans Briegel and Gorka De las Cuevas, which stores experience in a network of clips connected by weighted edges. Projective Simulation includes a natural dissipation mechanism — a forgetting parameter that continuously pulls the edge weights back toward their initial values, driving the policy gradually toward uniform exploration. Two modifications were required to make the hybrid version work in a changing world. First, because executing k rounds of amplitude amplification consumes 2k+1 interaction episodes, the quantum agent updates its policy less frequently than a classical one; the researchers derived an equivalent update rule that scales the dissipation strength so that the hybrid agent forgets at the same rate as its classical counterpart, and they proved the equivalence formally by induction. Second, the agent’s internal lower-bound estimate of its success probability — needed to calibrate the number of Grover iterations — had to be maintained adaptively, purging previously rewarded action sequences from memory whenever they were discovered to be no longer rewarded after an environment change.
The testbed was a modified Gridworld with a moving target. In the first scenario, the reward moves along a fixed path within each episode, synchronously with the agent’s own steps, adding time dependence within episodes while keeping the overall reward structure stable. In the second, more demanding scenario, the reward path itself is swapped for a new, fully disjoint path partway through training — a hidden-mode Markov decision process in which the agent knows a switch may occur but is never told when. Crucially, the researchers designed the second layout so that the two reward paths share no rewarded action sequences at all, making it the harshest possible test: excellence on the first path maximally damages performance on the second.
The results, averaged over hundreds of simulated runs, are striking. In the fixed-path scenario, the hybrid agent found its first reward in an average of 17.9 episodes, compared with 58.4 episodes for classical Projective Simulation — better even than the theoretical quadratic-speedup upper bound of 34.4 episodes — and after just 100 training episodes it had already reached an average success probability of 99.3 percent, while the classical agent lagged at 69.3 percent. Classical Q-Learning, with its one-step back-propagation of reward information, needed far longer for its values to percolate back to the starting state.
The changing-path scenario delivered the study’s most interesting twist. Immediately after the reward path switched, the hybrid agent’s success probability collapsed by roughly an order of magnitude below that of the classical agent — exactly the vulnerability the researchers had anticipated. But the forgetting mechanism did its work. The hybrid agent’s curve then climbed steeply, converging to nearly 100 percent success, while classical Projective Simulation rose more gradually to 98.4 percent and Q-Learning, hampered by its fixed exploration rate, plateaued between 60 and 80 percent. With a dissipation value of 0.05, the hybrid agent achieved the shortest total learning time in the benchmark — about 113 episodes against 149 for Q-Learning and 187 for the best classical Projective Simulation configuration. The researchers also showed that their adaptive estimate of the success probability tracks the true value closely, overshooting only marginally in the first few episodes after a switch thanks to the purging mechanism.
“Given that the problem scenario in this article is chosen such that the hybrid agent’s initial good performance puts it at a maximal disadvantage after a change in the reward function, we expect the results of this toy problem to generalize well to a wider range of problem cases,” the authors write. They go further, conjecturing that the quadratic speedup in learning may itself be what enables faster adaptation — potentially opening the door to learning in fast-changing environments where a classical agent would be too slow to learn at all.
The work sits within the broader and still-uncertain landscape of quantum reinforcement learning. Much current research in the noisy intermediate-scale quantum era relies on variational quantum circuits as function approximators, but whether those approaches can deliver genuine quantum advantage remains an open question. The hybrid agent, by contrast, carries a proven speedup and has already been demonstrated in a proof-of-principle experiment on a nanophotonic processor, suggesting it may be a serious contender for the post-NISQ era. The authors caution that their dissipation mechanism is deliberately passive — it does not detect changes but absorbs them — and that combining the hybrid agent with continual-learning methods that prevent catastrophic forgetting, or extending the study to time-dependent transition dynamics, are natural next steps. They also call for standardized continual-RL benchmarks, perhaps as extensions of the popular Gymnasium package, to put such comparisons on firmer footing.
For a field often accused of chasing speedups on toy problems that never change, the message of this study is pointed: quantum-accelerated learners can not only learn faster, but also unlearn faster. In a world where the only constant is change, that may be the advantage that matters.
Subject of Research: Hybrid quantum-classical reinforcement learning in non-stationary (dynamic) environments
Subject of Research: Technology and Engineering
Article Title: Quantum reinforcement learning in dynamic environments
Article References: Sefrin, O., Radons, M., Simon, L., & Wölk, S. (2026). Quantum reinforcement learning in dynamic environments. Quantum Machine Intelligence, 8(1), Article 58. https://doi.org/10.1007/s42484-026-00383-8
Image Credits: AI Generated
DOI: 10.1007/s42484-026-00383-8
Keywords: quantum reinforcement learning, hybrid algorithm, continual reinforcement learning, amplitude amplification, non-stationary environments, Projective Simulation, Gridworld, Grover search, sample complexity, forgetting mechanism, Markov decision process, quantum machine learning
Cite Scienmag News
APA MLA Chicago
Katie Riggs. (September 8, 2026). Quantum reinforcement learning adapts to changing environments. Scienmag. https://scienmag.com/quantum-reinforcement-learning-adapts-to-changing-environments/
Katie Riggs. “Quantum reinforcement learning adapts to changing environments.” Scienmag, 8 September 2026, https://scienmag.com/quantum-reinforcement-learning-adapts-to-changing-environments/. Accessed 8 September 2026.
Katie Riggs. “Quantum reinforcement learning adapts to changing environments.” Scienmag. September 8, 2026. https://scienmag.com/quantum-reinforcement-learning-adapts-to-changing-environments/
Copy citation Download RIS
Tags: adaptive AI in changing environmentsadaptive algorithmsdynamic environment adaptationhybrid quantum-classical machine learninghybrid quantum-classical reinforcement learningnon-stationary environment adaptationnon-stationary environmentsnon-stationary reinforcement learningoutpacing classical AI with quantum methodsquantum advantage in evolving scenariosquantum algorithms for changing environmentsquantum algorithms for shifting environmentsquantum machine intelligencequantum machine learning for dynamic settingsquantum machine learning researchQuantum reinforcement learningquantum reinforcement learning researchquantum-enhanced learningquantum-enhanced learning algorithmsreal-world application of quantum RLreal-world applications of quantum AIsystematic study of hybrid quantum RL agents
