Synthetic brain signals are becoming a serious currency in neuroscience, but judging whether a computer-generated electroencephalogram is actually any good has remained stubbornly subjective. A team of researchers now reports an open-source Python library, called seege_, that organises twenty established metrics into a single, interpretable workflow for evaluating synthetic EEG and intracranial EEG data. Rather than proposing brand-new statistics, the framework arranges complementary measures into three dimensions—fidelity, diversity and privacy—and translates them into composite scores bounded between zero and one, with higher values indicating closer agreement with real data. The work, published in Discover Neuroscience, aims to replace the patchwork of ad-hoc checks that currently dominates the field with a transparent, reproducible standard.
The motivation stems from a practical bottleneck. Clinical neurophysiological recordings, particularly intracranial EEG, are invasive, complex to acquire and restricted to small patient cohorts. Generative models such as Generative Adversarial Networks and Variational Autoencoders have been widely applied to synthesise EEG data to expand datasets, train more robust machine learning models and promote reproducibility. Yet the authors argue that progress is constrained by the absence of consistent, physiologically grounded evaluation practices. Current assessment strategies often rely on isolated statistical, visual or downstream-task metrics, which may not fully capture the temporal, spectral, nonlinear and spatial properties that characterise real EEG signals.
Why does this matter clinically? Key characteristics of EEG—spectral content, temporal dynamics and multifractal properties—are directly linked to the pathophysiological phenomena underlying brain activity, including spike patterns and high-frequency oscillations that are essential for diagnostic and research applications in epilepsy, sleep disorders and cognitive neuroscience. Existing evaluation metrics often fall short in capturing these nuanced, domain-specific attributes, limiting their ability to validate synthetic data for critical applications such as epilepsy monitoring or brain-computer interfaces. A synthetic signal can look statistically plausible in a histogram while missing the transient oscillatory structure that a neurologist would care about.
The seege_ framework assesses fidelity across five domains: time, frequency, time-frequency, complexity and spatial analysis. Time fidelity uses Hjorth activity, mobility and complexity descriptors together with Wasserstein distances and a Mahalanobis distance between feature vectors. Frequency fidelity examines relative power in canonical bands from delta at 0.5–4 Hz up to fast ripple at 250–500 Hz, dominant frequency, spectral coherence and a spectral Wasserstein distance computed with the Sinkhorn algorithm. Time-frequency fidelity compares continuous wavelet transform scalograms using colour structural similarity, root mean squared error and cosine similarity, complemented by burst statistics that extract features such as burst count, duration, rate and inter-burst interval from band-limited envelopes.
Complexity fidelity draws on multifractal detrended fluctuation analysis, detrended cross-correlation analysis and its multifractal extension, alongside sample entropy, permutation entropy and Lempel-Ziv complexity. Spatial fidelity evaluates whether synthetic signals preserve the cross-channel dependency structure intrinsic to multivariate EEG recordings, quantifying dissimilarity between channel correlation matrices using a Frobenius distance normalised by the variability of real-to-real comparisons. Diversity is evaluated through coverage, geometric structure and intrinsic variability, including nearest-neighbour relationships, PCA and UMAP embeddings, and ratios that flag both mode collapse and over-dispersion. Privacy is examined using nearest-neighbour similarity with Euclidean and dynamic time warping distances, plus membership-inference risk under a black-box attack framework.
A distinctive design choice is interpretability. Distances are normalised by the standard deviation of the corresponding real-data distributions, so metric magnitudes can be read within a standardised effect-size framework rather than as raw numbers. The library also produces domain-level composite scores, visual summaries such as histogram overlays and side-by-side scalograms, and interpretation thresholds contextualised against real-data variability. The authors emphasise that the dimensions are interdependent: perfect fidelity would demand near-exact replication of individual records, posing an unacceptable disclosure risk, while stringent privacy preservation or excessive diversification can obscure the underlying statistical structure and degrade downstream utility. A rigorous assessment must therefore evaluate fidelity, diversity and privacy jointly to reveal the trade-offs that emerge when tuning generative models.
To demonstrate the workflow, the team generated synthetic intracranial EEG with an attention-based Deep Convolutional Wasserstein GAN trained on pathological segments from the Mayo Clinic sEEG/ECoG dataset, which contains two-hour recordings from 25 patients with drug-resistant epilepsy undergoing presurgical evaluation. Validation was conducted on 200 segments comprising physiological, pathological and noise recordings using three complementary approaches: distribution-level discrimination under a real-real, real-synthetic and synthetic-synthetic pairing protocol with phase-randomised surrogates as negative controls; controlled perturbation experiments quantifying metric responses with within-signal effect sizes and bootstrap 95% confidence intervals; and quality-control comparisons spanning six interpretable conditions, from GAN synthesis to noise, mode collapse and memorisation.
The validation results were revealing. Commonly used literature metrics showed substantial heterogeneity and weak discrimination: cosine similarity collapsed across conditions, Jensen-Shannon divergence reversed the expected ordering, and Euclidean distance, despite statistical significance, yielded a negligible effect size. In contrast, the proposed metrics demonstrated consistent real-real versus real-synthetic separation across domains, with effect sizes ranging from moderate to large. Spectral coherence and structural similarity showed strong discrimination, while entropy-based measures exhibited large deviations and clear separation from phase-randomised controls. In the quality-control tests, the framework correctly identified each failure mode: the GAN condition yielded a global fidelity score of 0.526, with low time-domain (0.404) and complexity (0.353) agreement but moderate-to-high frequency (0.645) and time-frequency (0.835) scores; diversity remained good at 0.829 and privacy protection was maximal at 1.000. The noise condition produced the lowest fidelity agreement, diversity collapsed under mode collapse at 0.534, and a memorised real control achieved near-perfect fidelity and diversity but a privacy score of zero.
The authors are candid about limitations. The empirical validation is restricted to the Mayo Clinic intracranial dataset and a single GAN-based generator, and the results should be read as methodological validation of the evaluation framework rather than evidence of clinical validity across EEG applications generally. Two metrics were not evaluated in the case study: spatial fidelity requires multivariate recordings and was not applicable to the single-channel segments used, and membership-inference risk requires knowledge of which samples were excluded from model training, which was unavailable, so privacy scores reflected nearest-neighbour distances only. Two open questions remain: whether higher framework scores predict better downstream performance has not been tested, and the framework currently demonstrates statistical rather than confirmed biological realism, since neither entropy measures nor the burst-detection module have been validated against expert-annotated biomarkers such as epileptiform spikes, high-frequency oscillations or sleep spindles.
The library, released as open-source code on GitHub with the analysed dataset available in a public repository, is intended to support transparent reporting and comparison of synthetic EEG quality across generative approaches. The authors note that the computational burden of certain analyses, such as complexity fidelity and privacy evaluation, may limit suitability for real-time deployment and typical personal computers, raising emerging concerns about computational sustainability; future work will prioritise algorithmic optimisation and scalable implementations, alongside validation on broader neurophysiological datasets and generative families. For a field increasingly reliant on synthetic data to sidestep the scarcity of invasive human recordings, a shared yardstick for quality—however imperfect its current reach—marks a concrete step toward benchmarks that researchers, reviewers and clinicians can actually compare.
Subject of Research: An open-source evaluation framework for assessing the quality of synthetic EEG data
Article Title: Interpretable metrics for evaluating fidelity, diversity, and privacy in synthetic EEG data
Article References: Silveira, I., Silva, L., Furk, D., Marques, B., Ao, N., Hoogteijling, S., Heijink, I., Zijlmans, M., & Gamboa, H. (2026). Interpretable metrics for evaluating fidelity, diversity, and privacy in synthetic EEG data. Discover Neuroscience, 21(1), Article 96. https://doi.org/10.1186/s13064-026-00323-z
Image Credits: AI Generated
DOI: 10.1186/s13064-026-00323-z
Keywords: synthetic EEG, intracranial EEG, generative adversarial networks, fidelity metrics, data privacy, membership inference, open-source software, epilepsy, neuroscience, machine learning, time-series evaluation, Python library
News Source: Cassandra Pierce. (October 30, 2026). New open-source toolkit grades synthetic brain recordings for fidelity, diversity and privacy. Scienmag.
