Artificial intelligence systems are increasingly being asked to do something that sounds paradoxical: forget. As regulators around the world tighten data protection rules and the public grows wary of how personal information is used to train machine learning models, researchers have been racing to develop techniques that allow a trained neural network to expunge specific knowledge without the enormous expense of retraining from scratch. A new study published in the journal Machine Learning takes a significant step toward making that possible in one of the hardest settings imaginable—continual learning, where a model must keep absorbing new tasks over time while its access to old data slips away.
The research, led by Sayanta Adhikari, Vishnuprasadh Kumaravelu, and P. K. Srijith of the Bayesian Reasoning and Inference Lab at the Indian Institute of Technology Hyderabad, introduces a framework called UnCLe, short for a Hypernetwork Framework for Data-Free Unlearning and Continual Learning. The core insight is that machine unlearning—the deliberate removal of a task’s influence from a trained model—has almost always been designed with offline training in mind, where engineers retain full access to the original dataset. In continual learning, that assumption collapses. Data arrives task by task and is typically discarded after use, so when an unlearning request arrives, the original examples may simply no longer exist.
The team identified two failure modes that emerge when conventional unlearning is naively applied to continual learning environments. The first is catastrophic forgetting of retained tasks, a well-known pathology in which updating a network to remove one capability wipes out unrelated capabilities it was supposed to keep. The second is subtler and, in some ways, more troubling: catastrophic remembering, in which tasks that were supposedly unlearned resurface when the model later absorbs new information. A model that appears to have forgotten sensitive data can effectively relapse, undermining the very privacy guarantees that unlearning is meant to provide.
UnCLe attacks both problems by restructuring how the model’s parameters are produced in the first place. Instead of training a single monolithic network, the framework employs a hypernetwork—a network that generates the weights of another network—conditioned on compact task embeddings. Each task the system encounters is represented by its own embedding vector, and the hypernetwork maps that embedding to a full set of task-specific parameters. Learning a new task therefore means learning or refining an embedding, while the shared hypernetwork machinery remains stable across the entire sequence of operations.
Unlearning under this scheme becomes elegantly simple at the task level. To remove a task, the framework optimizes the hypernetwork so that, for that task’s embedding, it generates parameters that behave like noise. The generated network produces uniform, maximum-entropy outputs on the forgotten task—in other words, the model becomes maximally uncertain, exactly as if it had never seen the task at all. Crucially, this procedure does not require the original training data, which is precisely what makes it data-free. The optimization is guided by a mean squared error objective that pulls the generated parameters toward freshly sampled Gaussian noise, combined with a regularization term that anchors the hypernetwork’s outputs for all previously retained tasks, preventing collateral damage.
The choice of a noise-matching objective, rather than a direct norm penalty, turns out to matter a great deal. The authors show mathematically that averaging the squared distance to random Gaussian samples converges to the squared L2 norm of the parameters plus a constant, meaning the MSE objective implicitly drives the forgotten task’s parameters toward zero and its outputs toward a uniform distribution. But applying the L2 norm directly would, over repeated unlearning operations, drag the hypernetwork’s own shared weights toward zero and destabilize the whole system. By contrast, sampling a fresh noise target at each optimization step constrains the forget task only in distribution, acting as an implicit regularizer that preserves the shared representation. The researchers compared alternatives—including fixed noise targets, pure norm reduction, and simply discarding the task embedding—and found their approach achieved the best balance between erasing the target task and protecting retained performance.
Scaling a hypernetwork to generate all the weights of a modern convolutional backbone such as ResNet18 or ResNet50 presents its own engineering challenge, since the hypernetwork’s output layer would otherwise balloon to an impractical size. The team’s solution is chunked generation: the main network’s parameters are partitioned into roughly 200 chunks, each produced by a dedicated head of the hypernetwork conditioned on a unique chunk embedding concatenated with the task embedding. These chunk embeddings are frozen after the first task to guard against forgetting, and the final layer is split into specialized heads for weights, batch normalization parameters, and residual connection parameters, reducing redundancy and computational overhead.
The practical consequences of this design are striking. Because unlearning operates entirely in parameter space, its computational cost is dominated by the hypernetwork’s forward and backward passes and is essentially independent of dataset size and class count. The only term that grows over a sequence is the regularization over retained tasks, and it grows linearly in the number of tasks, not data points. In conventional replay-based unlearning adapted to continual settings, by contrast, the cost scales with a replay buffer whose size is difficult to budget in advance. The researchers also introduced an annealing strategy that shrinks the burn-in phase of each unlearning operation by ten percent per operation, exploiting forward transfer to cut unlearning time without degrading quality.
Empirical evaluations across sequential vision benchmarks—including Permuted-MNIST, a five-dataset suite combining MNIST, Fashion-MNIST, KMNIST, notMNIST and SVHN, CIFAR-100, and TinyImageNet—showed that UnCLe can perform long interleaved sequences of learning and unlearning requests, up to 30 operations on TinyImageNet, with minimal disruption to previously acquired knowledge. Measured against baselines including fine-tuning, retraining from scratch, and hypernetwork variants that rely on natural catastrophic forgetting, UnCLe performed on par or better across metrics, and strictly better on three of five measures with a ResNet-18 backbone. It also achieved membership inference attack accuracy closest to the ideal fifty percent, a key indicator that forgotten data is genuinely indistinguishable from never-seen data—a central concern for privacy.
Perhaps most importantly for real-world deployment, UnCLe prevented the relapse phenomenon that plagues conventional approaches: tasks unlearned by prior methods tend to creep back once new learning occurs, whereas UnCLe’s unlearned tasks stayed forgotten even as subsequent tasks were absorbed. The authors argue this has broad implications for responsible AI governance, from honoring the right to be forgotten under data protection law to stripping biased or harmful behaviors from deployed models without full retraining. At the same time, they caution that the very possibility of relapse under weaker methods underscores the need for robust verification mechanisms. With code released publicly, the framework offers a template for AI systems that can keep learning throughout their operational life while remaining accountable to demands that they forget on command.
Subject of Research: A hypernetwork framework enabling data-free machine unlearning within continual learning settings
Article Title: A Hypernetwork Framework for Data-Free Unlearning and Continual Learning
Article References: A Hypernetwork Framework for Data-Free Unlearning and Continual Learning. (n.d.). https://doi.org/10.1007/s10994-026-07125-8
Image Credits: AI Generated
DOI: 10.1007/s10994-026-07125-8
Keywords: machine unlearning, continual learning, hypernetworks, data privacy, catastrophic forgetting, task embeddings, neural networks, AI safety, right to be forgotten, ResNet, membership inference, parameter generation
Cite Scienmag News
APA
MLA
Chicago
Denise Maddox. (September 22, 2026). AI Learns to Forget: New Hypernetwork Framework Enables Data-Free Unlearning. Scienmag. https://scienmag.com/ai-learns-to-forget-new-hypernetwork-framework-enables-data-free-unlearning/
Denise Maddox. “AI Learns to Forget: New Hypernetwork Framework Enables Data-Free Unlearning.” Scienmag, 22 September 2026, https://scienmag.com/ai-learns-to-forget-new-hypernetwork-framework-enables-data-free-unlearning/. Accessed 22 September 2026.
Denise Maddox. “AI Learns to Forget: New Hypernetwork Framework Enables Data-Free Unlearning.” Scienmag. September 22, 2026. https://scienmag.com/ai-learns-to-forget-new-hypernetwork-framework-enables-data-free-unlearning/
Copy citation
Download RIS
Tags: AI safetycatastrophic forgettingcontinual learningData Privacyhypernetworksmachine unlearningmembership inferenceneural networksparameter generationResNetright to be forgottentask embeddings
