Hebbian Learning Shows How Brains Could Spend Their Synaptic Budget Wisely

Every thought, memory, and perception in the brain runs on a finite metabolic budget. Maintaining the trillions of synapses that link neurons together consumes a substantial share of the body’s energy, and the physical wiring of cortical circuits limits how many connections can exist in the first place. A new computational study argues that one of neuroscience’s oldest ideas about learning—Hebbian plasticity, often summarized as “neurons that fire together, wire together”—may be precisely the kind of mechanism a metabolically constrained brain needs, not because it maximizes accuracy, but because it allocates scarce synaptic resources with striking efficiency.

The study, published in Frontiers in Computational Neuroscience by Patrick Inoue, Florian Röhrbein, and Andreas Knoblauch, set out to test whether an excitatory, competitive Hebbian rule can implement what the researchers call synaptic resource allocation under anatomical and metabolic constraints. Rather than asking which learning rule produces the best classifier, the team asked a subtler question: for a given level of task-relevant information, how much irrelevant information about the input does each rule force the network to retain? The answer, they found, favors Hebbian learning in a way that standard backpropagation and biologically inspired alternatives do not.

To make this question measurable, the researchers turned to information theory. Drawing on the Variational Information Bottleneck framework, which formalizes the trade-off between compressing sensory input and preserving information relevant to a task, they defined a metric they call task-information cost, or CTI. This quantity compares the mutual information a network’s latent code retains about its input with the mutual information it retains about the correct answer. Values close to one indicate a lean, efficient code; large values indicate a network that hoards input details it does not need. The authors are careful to note that CTI is a comparative proxy for representational burden, not a direct measurement of biological energy expenditure, but its logic is grounded in the thermodynamic insight that sustaining more distinct internal states requires more physical work.

The experimental design was deliberately controlled. The team used three audiovisual benchmarks—AVE, Kinetics-Sounds, and VGGSound100—representing video clips labeled with the sounds and events they contain. Instead of training networks on raw pixels and waveforms, they fed all models fixed embeddings produced by pretrained VideoMAE and AudioMAE encoders, treating these as stand-ins for high-level sensory representations already computed by earlier processing stages. This choice isolates the downstream associative plasticity under study: any differences in representational efficiency can be attributed to the learning rule itself rather than to quirks of feature extraction. Networks were built as fully connected multilayer perceptrons in shallow and five-hidden-layer variants, with layer widths matched to the input dimensionality.

The Hebbian rule under investigation builds on Oja’s classic neural principal component analysis rule, extended with nonnegativity constraints and normalization. Its update combines two opposing influences: a synapse-specific potentiation term that strengthens connections when presynaptic and postsynaptic activity coincide, and a competitive decay term computed at the level of the presynaptic neuron and broadcast to its outgoing synapses. This separation breaks the practical symmetry of the algebra, strengthening correlated inputs while decorrelating synapses that project to multiple targets. The result is a weight matrix that is sparse, nonnegative—consistent with Dale’s law, the principle that a neuron’s outputs are uniformly excitatory or inhibitory—and implicitly performs something like principal component analysis on its inputs, concentrating representational capacity onto a low-dimensional, decorrelated subspace.

For comparison, the researchers trained identical architectures with standard backpropagation and with Dense Difference Target Propagation, a biologically motivated alternative that avoids the weight-transport problem of backpropagation by propagating layer-wise targets rather than error gradients. To ensure fairness, all networks were subjected to the same sparsification procedure: after 500 epochs of training, layer-wise magnitude pruning removed the least important weights, targeting roughly 90 percent sparsity, a figure consistent with estimates of cortical connectivity. This dense-to-sparse strategy also mirrors development, since cortical circuits are initially overconnected and are pruned by activity-dependent processes as the brain matures.

The results were striking in a specific and somewhat counterintuitive way. Hebbian-trained networks did not uniformly win on classification accuracy. On some datasets, backpropagation and DDTP reached higher top-1 accuracy. But when the researchers examined the cost side of the ledger, the Hebbian models consistently occupied a low-CTI regime. In the main compressed comparisons, sparse backpropagation and DDTP showed geometric CTI ratios of 5.37 and 6.73 relative to the Hebbian model, differences that remained statistically significant after correction for multiple comparisons. In one illustrative shallow-network setting on VGGSound100, dense backpropagation, sparse backpropagation, and DDTP achieved closely matched accuracies of around 62 to 63 percent, yet their CTI values ranged from 25 to 100, while the Hebbian model reached 60.7 percent accuracy at a CTI of just 21.

Ablation experiments sharpened the picture. When the researchers pruned trained networks further without retraining, Hebbian models were remarkably robust: accuracy remained stable across a broad range of sparsity and degraded only when connectivity fell below roughly 10 percent, the point at which pruning began to eat into the functional support of the representation. Backpropagation and DDTP networks, by contrast, showed measurable degradation as pruning increased, suggesting that their performance depends on globally coordinated synaptic configurations rather than on a locally assigned, sparse set of connections. Control experiments also showed that the Hebbian rule could produce compact, task-relevant representations from raw pixel inputs on MNIST, reaching 91.7 percent accuracy with a CTI of 12.1, though the main audiovisual findings concern allocation over fixed upstream embeddings.

The authors interpret these findings through the lens of layer-wise variance redistribution. Successive Hebbian layers progressively decorrelate their outputs while concentrating capacity on task-relevant dimensions, effectively implementing a local, PCA-like decomposition of the input distribution. Early layers capture dominant modes of the input; later layers encode residual variance. This structure limits retained information about the input while preserving what matters for the task. The researchers also note that the apparently global competitive correction term in their update rule could, in principle, be realized locally through recurrent inhibition or dendritic voltage signals, connecting the algorithm to plausible cortical circuitry.

The study has clear boundaries. A ten-hidden-layer control configuration proved unstable under the compressed evaluation setting, indicating that the efficiency gains do not automatically extend to greater depth. The information-theoretic cost metric neglects contributors to neural energy expenditure such as spike-timing variability and glial support, and the audiovisual results concern allocation over pretrained embeddings rather than end-to-end feature discovery. Still, the central message stands: Hebbian learning is best understood not as a general-purpose accuracy maximizer but as a local mechanism for synaptic resource allocation—one that shifts the trade-off between task-relevant information and representational cost in a direction that a metabolically frugal brain would plausibly favor. In an era when artificial networks are routinely overparameterized, the finding offers a reminder that biological intelligence may be defined as much by what it declines to represent as by what it computes.

Subject of Research: Computational study of constrained Hebbian learning as a local mechanism for efficient synaptic resource allocation under metabolic and anatomical constraints

Article Title: Constrained Hebbian learning supports efficient representational allocation under structural constraints

Article References: Constrained Hebbian learning supports efficient representational allocation under structural constraints. (n.d.). https://doi.org/10.3389/fncom.2026.1825378

Image Credits: AI Generated

DOI: 10.3389/fncom.2026.1825378

Keywords: Hebbian learning, synaptic plasticity, information bottleneck, computational neuroscience, neural efficiency, sparsity, metabolic constraints, audiovisual learning, Oja's rule, representational cost, Dale's law, neural networks