New Toolkit Puts Simulated Neural Circuits to the Test

…

The central premise behind the Computation-through-Dynamics Toolkit rests on a conceptual shift that has been gaining momentum in systems neuroscience over the past decade: rather than treating populations of neurons as collections of independent units, researchers increasingly describe neural computation as the evolution of activity through a low-dimensional dynamical system. In this view, the trajectory of neural state over time—shaped by recurrent interactions within the circuit and by external inputs—carries the computational content that ultimately drives behavior. This framing has proven especially productive for understanding cognitive tasks such as working memory, decision-making, and timing, where sustained or slowly evolving activity patterns appear to implement the underlying computation. Because the dynamical rules themselves cannot be observed directly from recordings, the field has come to rely on latent dynamical models, including state-space models and recurrent neural network approaches, that attempt to recover these rules from noisy, high-dimensional measurements of spiking or other signals.

A persistent obstacle in this enterprise has been the question of validation. When a model is fit to real neural data, there is no ground truth against which the inferred dynamics can be compared. The model may reproduce the observed activity faithfully while still misrepresenting the underlying circuit mechanisms, a problem analogous to overfitting at the level of dynamics rather than single neurons. Synthetic datasets with known generative structure offer a way around this impasse, since the true dynamics are available by construction and model performance can be measured against them. Yet the synthetic benchmarks historically available to the community have often been too simple to serve as meaningful proxies. Linear systems, low-dimensional attractors, or networks of randomly connected units fail to capture features that many investigators consider essential to biological computation, such as heterogeneous timescales, structured connectivity, input-driven transitions between discrete dynamical regimes, and the nonlinear gating of information flow. A model that performs well on such simplified benchmarks may nonetheless fail when confronted with the messier dynamics of real circuits, and the field has had little systematic way to know which failures matter.

The datasets assembled in the toolkit are designed to close this gap by embedding computational properties that recur across biological neural circuits. Rather than treating synthetic data as mere placeholders for real recordings, the toolkit treats them as carefully constructed test cases in which specific aspects of neural computation—fixed-point dynamics supporting memory maintenance, input-dependent switching between attractor states, transient responses to stimulus onsets, and the flexible input-output mappings characteristic of trained behavior—are instantiated with known parameters. This design philosophy mirrors the logic of controlled experiments in the laboratory: by holding most features constant and varying a targeted property, a benchmark can isolate precisely which capabilities a model lacks. For developers of latent dynamical models, this means that failures on particular datasets can be traced to specific mechanistic shortcomings rather than to an undifferentiated sense that the model performs poorly.

The second pillar of the toolkit, the set of evaluation metrics, addresses a problem that has quietly hampered progress in the field. Traditional measures of model quality in neuroscience have largely focused on how well a model predicts or reconstructs recorded activity, using quantities such as per-trial firing rate error or bits per spike. These measures are useful but incomplete, because a model can achieve excellent single-step prediction while its inferred latent dynamics diverge substantially from the true system over longer horizons. Two models with similar reconstruction scores may differ dramatically in whether they capture the attractor structure, the timescales, or the input sensitivity of the underlying circuit. The metrics introduced here are interpretable in the sense that they target specific, well-defined properties of dynamics—such as the accuracy of recovered fixed points, the fidelity of inferred timescales, and the degree to which the model’s response to inputs matches the ground-truth system—rather than relying on a single aggregate score. This decomposition allows developers to diagnose not just whether a model is good, but in what ways it falls short.

The standardized pipeline constitutes the third component, and its importance should not be underestimated. In a rapidly growing field where new modeling approaches appear frequently, comparisons across papers are often confounded by differences in training procedures, hyperparameter settings, data preprocessing, and evaluation protocols. A model that appears superior in one study may simply have been trained more carefully or evaluated more favorably. By providing a common infrastructure for training and evaluation, the toolkit reduces this source of noise and makes head-to-head comparisons meaningful. The pipeline accommodates models both with and without access to known external inputs, reflecting an important distinction in the field: some latent dynamical models assume that the experimenter knows the stimuli delivered to the animal and can condition the inferred dynamics on them, while others must infer the influence of unobserved inputs from the activity alone. Providing infrastructure that supports both regimes broadens the toolkit’s applicability across the diverse family of models now in use.

The practical value of this framework becomes clearest when considering the iterative process of model development. Building a latent dynamical model involves numerous design decisions—architectural choices, dimensionality of the latent space, regularization schemes, inference procedures—each of which can affect the recovered dynamics in subtle ways. Without principled diagnostics, developers often rely on intuition and trial-and-error, a process that is slow and prone to confirmation bias. The toolkit’s metrics function as a form of instrumentation, analogous to the measurement devices that guide experimentation in the wet lab. A developer can tune a model, run it through the standardized evaluation, and receive targeted feedback about which aspects of the dynamics have improved and which have degraded. This turns model development from an art into something closer to an engineering discipline, with measurable objectives and reproducible benchmarks.

The relationship between synthetic benchmarks and real neural data deserves careful consideration. Synthetic datasets are not meant to replace real recordings but to complement them, occupying a middle ground between pure theory and empirical application. Theory can prove properties of idealized systems, and real data can reveal what models achieve in practice, but neither alone can answer the question of whether a model’s apparent success on real data reflects genuine recovery of dynamics or merely a good fit to surface statistics. Synthetic data with known ground truth occupies exactly this middle ground, and the more faithfully such data reflect the computational properties of biological circuits, the more informative the transfer from benchmark performance to real-world expectations becomes. The emphasis in the toolkit on incorporating features of biological computation—rather than mathematically convenient abstractions—reflects a deliberate effort to maximize this transfer.

It is also worth situating this work within the broader trajectory of dynamical systems thinking in neuroscience. The idea that neural populations implement computations through their dynamics has deep roots, tracing back to early theoretical work on attractor networks and continuous recurrent circuits, and it has been revitalized in recent years by advances in recording technology, machine learning, and latent variable modeling. As these threads have converged, the field has produced an expanding zoo of methods for extracting dynamics from data, each with different assumptions and trade-offs. What has been missing is a shared foundation for evaluating these methods—a common set of test cases and metrics that the community agrees upon. The absence of such a foundation has real costs: results that cannot be compared across studies, methods that are adopted on the basis of anecdotal performance, and progress that is difficult to accumulate. Standardized benchmarks have historically played a transformative role in other areas of computational science, and the introduction of a rigorous benchmarking framework for neural dynamics models follows a pattern that has repeatedly accelerated field-wide progress.

For experimentalists, the implications of this work are subtler but still significant. Models that accurately recover the dynamics of a circuit can generate hypotheses that are difficult to obtain from raw recordings alone, such as predictions about how perturbations to specific inputs will alter the trajectory of neural state and, consequently, behavior. The reliability of such hypotheses depends on the fidelity of the inferred dynamics, which is precisely what the toolkit’s metrics are designed to quantify. As experimentalists begin to demand evidence that computational models capture dynamical structure rather than merely surface statistics, tools for establishing that evidence become part of the essential vocabulary between theory and experiment.

Looking forward, the framework opens several avenues for continued development. As new modeling approaches emerge—whether drawing on advances in machine learning, statistical inference, or theoretical neuroscience—they can be evaluated against the same benchmarks, enabling the community to build a cumulative picture of which methodological choices matter and under what conditions. The datasets themselves can be extended to capture additional computational motifs as they are identified in biological circuits, and the metrics can be refined as the field develops sharper intuitions about what constitutes faithful recovery of dynamics. In this way, the toolkit is best understood not as a finished product but as infrastructure: a foundation on which the community can construct increasingly rigorous and comparable studies of how neural circuits compute. By making the invisible rules of neural dynamics testable, it brings the field one step closer to explaining, rather than merely describing, how ensembles of neurons give rise to goal-directed behavior.

Subject of Research: …

Article Title: Computation-through-Dynamics Toolkit: Simulated datasets and quality metrics for dynamical models of neural activity

Article References: Versteeg, C., McCart, J. D., Ostrow, M., Zoltowski, D. M., Washington, C. B., Driscoll, L., Codol, O., Michaels, J. A., Linderman, S. W., Sussillo, D., & Pandarinath, C. (2026). Computation-through-Dynamics Toolkit: Simulated datasets and quality metrics for dynamical models of neural activity. PLOS Computational Biology, 22(10), e1014755. https://doi.org/10.1371/journal.pcbi.1014755

Image Credits: AI Generated

DOI: 10.1371/journal.pcbi.1014755

Keywords: neuroscience, computational modeling, neural dynamics, machine learning, toolkit, synthetic data, PLOS Computational Biology, systems neuroscience, neural networks, model validation, Computation-through-Dynamics, Simulated