Anomaly detection has become one of the quiet workhorses of modern computing. Banks use it to flag suspicious transactions, hospitals use it to spot unusual patient readings, and industrial systems use it to catch machines drifting toward failure. But there is a persistent frustration baked into all of these systems: they can tell you that something is strange, yet they rarely tell you why. When an algorithm flags a record as anomalous, analysts are often left staring at dozens or hundreds of features, wondering which of them actually drove the alarm. A new study published in the journal Machine Learning tackles that gap head-on with a framework called RLAX, which turns the search for an explanation into a reinforcement learning problem that an artificial agent learns to solve.
The work, carried out by Simone Amirato, Fabrizio Angiulli, and Fabio Fassetti of the University of Calabria in Italy, addresses what the authors call the Anomaly Explanation problem. Their idea is deceptively simple: an explanation of an anomaly is a compact subset of features, a subspace, in which the anomalous examples deviate most sharply from the normal population. If a fraud-detection system flags a transaction, the explanation might boil down to three or four attributes, such as transaction amount, merchant category, and time of day, that together make the record look nothing like ordinary behavior. The difficulty is that finding the best such subset is a combinatorial nightmare. With d features, the number of possible subsets grows as the power set of the feature set, a search space so vast that exhaustive enumeration is hopeless even for moderately sized problems.
RLAX, short for Reinforcement Learning-based Anomaly eXplanation, recasts this search as a Markov Decision Process. Each state in the process is a candidate subspace, and each action corresponds to adding a single feature to the current selection. The agent starts from the empty set and progressively builds up an explanation, one feature at a time. What guides it is a reward signal derived from a quantity the authors call the Subspace Margin. This margin measures the difference between the anomaly scores of known outlier examples and known inlier examples within the current subspace, normalized by the square root of the subspace size. The anomaly score itself is a classic distance-based measure: the average distance from a point to its K nearest neighbors, which serves as a proxy for local density. When the margin grows after adding a feature, the agent earns a positive reward; when the margin shrinks, the reward is negative.
To learn a policy over this enormous action space, the team turned to Proximal Policy Optimization, or PPO, a policy-gradient algorithm that has become a standard tool in the reinforcement learning repertoire. The choice is deliberate. Value-based methods such as Deep Q-Networks struggle when the number of actions is very large, and in RLAX the number of actions equals the number of features, which can easily reach into the hundreds or thousands in real applications. Policy gradient methods, by contrast, handle large action spaces gracefully. The authors used the Stable-Baselines3 implementation of PPO with a custom actor-critic policy that masks inadmissible actions, keeping the default architecture of two 64-unit tanh layers for both the policy and value networks. Once training is complete, the learned policy is distilled into a ranking of features, either through the empirical mean of discounted rewards each action collected during training or directly from the policy’s action probabilities, and candidate subspaces are read off from the top of that ranking.
A key design element is the Explanation Gain, a measure that trades off separation accuracy against explanation size. Formally, the gain of a subspace is its separation quality minus a penalty proportional to the fraction of features it uses, with a tunable coefficient alpha that defaults to 0.5. This penalty is what pushes the agent toward compact explanations. The experimental analysis shows why it matters: the subspace that maximizes raw separation, measured by the Area Under the ROC Curve, may require more than 30 percent of the available features, whereas the best-gain solution often involves only a small fraction of them while preserving comparable discriminative performance. In other words, the framework resists the temptation to throw every feature at the problem and instead delivers explanations an analyst can actually read.
The journal version of the work extends an earlier conference paper with a Subspace Refinement Strategy that makes the exploration genuinely non-monotone. In addition to add actions, the agent can now remove features it previously selected or issue a stop action to terminate the episode early. This flexibility matters for two reasons. First, remove actions let the agent backtrack out of locally suboptimal regions of the search space, revising earlier selections that looked promising in isolation but hurt in combination. Second, the stop action prevents the agent from wandering through suboptimal territory merely to exhaust its step budget. The authors impose a budget on add operations to prevent loops, but leave remove operations unconstrained since they are inherently bounded by the number of prior additions. Episodes can end through the stop action, budget exhaustion, or a patience criterion that triggers after several consecutive negative rewards.
The empirical evaluation spans benchmark datasets from the ODDS repository, including arrhythmia with 274 features, mnist with 100 features, and several others, alongside two substantially larger real-world datasets: Internet Advertisements with 1,558 features and Swarm Behaviour with 2,400 features. Against competitors including V-GAN, SOD, HiCS, a genetic subspace-selection method, and random feature selection, RLAX achieved the highest Explanation Gain on every benchmark dataset, indicating the most favorable balance between discriminative power and compactness. Statistical significance against random selection was confirmed with one-sided paired t-tests adjusted using Holm’s correction. The refinement strategy also proved its worth in training dynamics: on representative datasets it produced markedly shorter episodes, saving roughly 30 percent of execution time while achieving improved AUC at the same feature subset sizes. On synthetic scalability tests, RLAX’s runtime grew only moderately, by factors of roughly 1.5 to 1.9 as the number of instances increased twentyfold, a growth rate substantially lower than that of the genetic and SOD baselines.
Perhaps the most striking evidence for the framework’s practical value comes from a qualitative case study on the cardio dataset, which contains cardiotocography measurements of fetal heart rate with well-established clinical semantics. For a representative anomaly, RLAX selected four top features: the number of prolonged decelerations per second, the histogram variance of the fetal heart rate, the percentage of time with abnormal long-term variability, and the number of fetal movements per second. Each of these maps directly onto recognized indicators of fetal distress in the obstetric literature. Prolonged decelerations are documented as strong markers of potential fetal hypoxia and acidosis, abnormal long-term variability is one of the four fundamental characteristics used to interpret a cardiotocography trace, reduced fetal movements are a classic clinical warning sign, and heart rate variance captures deviations in oscillation amplitude in either direction. An algorithm with no medical knowledge, in other words, recovered a clinically coherent explanation.
The authors are careful about the limits of their approach. Because RLAX explores the search space through single-feature modifications, strongly non-additive interactions between features, where an informative combination only pays off after several individually unrewarding additions, can be harder to discover, though the remove actions partially mitigate this. They also note that the method assumes access to a small set of labeled anomalies and inlier examples, which in practice may come from a detector’s output or from a domain expert. Future work, they write, will explore domain-informed reward shaping, alternative separation measures, and settings with heterogeneous features and weaker supervision. The source code has been released publicly, and all datasets used in the study are openly available.
What makes RLAX more than a technical curiosity is the broader shift it represents in how machine learning systems might communicate their judgments. As automated decisions increasingly touch medicine, security, and environmental monitoring, the demand is growing not just for accurate detectors but for systems that can articulate the basis of a flag in terms a human can verify. By demonstrating that a reinforcement learning agent can navigate a combinatorially explosive search space and return explanations that align with clinical knowledge, the Calabria team offers a concrete template for that kind of accountability. The anomalies that matter most are often the ones we cannot explain away, and a tool that points precisely at the handful of features responsible turns a cryptic alarm into the beginning of an investigation rather than the end of one.
Subject of Research: A reinforcement learning framework for explaining anomalies through compact feature subspaces
Article Title: RLAX: A Reinforcement Learning-Based Framework for Explaining Anomalous Data
Article References: RLAX: A Reinforcement Learning-Based Framework for Explaining Anomalous Data. (n.d.). https://doi.org/10.1007/s10994-026-07182-z
Image Credits: AI Generated
DOI: 10.1007/s10994-026-07182-z
Keywords: reinforcement learning, anomaly explanation, explainable AI, feature selection, PPO, subspace margin, anomaly detection, machine learning, interpretable AI, outlier detection, cardiotocography, high-dimensional data
News Source: Denise Maddox. (October 13, 2026). AI Learns to Explain Why Data Goes Wrong, One Feature at a Time. Scienmag.
