AI Learns to Scrub Bias From Fake Data Before It Poisons Real Decisions

Synthetic data has quietly become one of the most consequential technologies in modern machine learning. In domains like finance, healthcare, and education, real records are locked behind privacy walls, governed by regulation, or simply too scarce to train reliable models. Generators that fabricate realistic fake records promise a way out: models can learn from statistically faithful stand-ins without exposing a single real patient or customer. But a new study published in Discover Artificial Intelligence warns that this promise carries a hidden cost. When the original data is biased, the synthetic copies tend to be biased too, and the machines that generate them can even amplify the skew. A research team from the Vietnam Academy of Science and Technology, Hanoi University of Science and Technology, the University of Koblenz, and the Banking Academy of Vietnam has now built a framework, called SENNA, that confronts this problem head-on by making fairness a design objective rather than an afterthought.

The core insight behind SENNA, short for Semantic ENcoding for fair tabular data geNerAtion, is that bias enters the pipeline at multiple points, so control must be applied at multiple points as well. The framework operates on tabular records, the rows of numbers and categories that dominate real-world datasets, from census tables to loan applications. Each record is first converted into a natural-language sentence using a fixed template, so that a row describing age, occupation, and income becomes a readable description. This conversion is purely representational and does not alter the underlying distribution, but it unlocks something powerful: pretrained language models can now encode the semantic relationships between attributes, rather than treating every column as an independent dimension the way conventional tabular encoders often do.

The technical heart of the method is disentangled representation learning. A frozen BERT encoder produces contextualized embeddings of each sentence, and two trainable projection heads then split this representation into two separate latent variables. The first, called the task-relevant variable, is trained to concentrate exactly the information needed to predict the target label, such as whether a loan will default. The second, the task-independent variable, is trained to forget the label entirely, using an adversarial classifier that tries to recover the label from it while a gradient reversal layer teaches the representation to confuse that classifier. The two variables are additionally pushed toward statistical independence with the Hilbert-Schmidt Independence Criterion, and a reconstruction objective prevents the latent space from collapsing into degenerate solutions. Variational encoders with Kullback-Leibler regularization keep the whole optimization smooth. The result is a latent space in which what matters for prediction and what does not are, at least approximately, separated.

This separation is then exploited at the generation stage, which is where large language models come in. Recent LLM-based tabular generators transform records into text and use in-context learning, presenting a handful of example records in the prompt and asking the model to produce new ones. The approach is flexible and expressive, but it is notoriously sensitive to which examples are chosen. When the in-context exemplars reflect demographic imbalance, the generated samples inherit the same skew, a phenomenon the authors describe as bias propagation. SENNA counters this by selecting exemplars not at random but through k-means clustering in the task-independent latent space. Because that space was trained to exclude label information, the selected exemplars cover diverse semantic territory without reinforcing spurious correlations between sensitive attributes and outcomes. The model is then prompted with a fixed template shared by training data, exemplars, and generated samples, and asked to produce candidate records that are converted back into tabular form.

Generation alone is not trusted. A two-stage post-generation process screens every candidate. First, individual fairness is enforced through nearest-neighbor validation: each generated sentence is encoded into the task-relevant space, its k nearest real neighbors are found, and a majority vote estimates the expected label. A candidate is retained only if its extracted label matches this local estimate, filtering out semantically inconsistent samples that would distort the decision boundary. Second, group fairness is enforced through balancing: the validated dataset is partitioned by protected-group membership and outcome, and subgroups are subsampled toward approximately uniform sizes, reducing residual correlations between sensitive attributes and predictions. Crucially, protected attributes are never used during representation learning or exemplar selection; they enter only at balancing and at evaluation, a design choice that limits how much sensitive information the pipeline itself consumes.

The empirical evaluation is unusually thorough for this area. The team compared SENNA against five established baselines, including the GAN-based CTGAN and TabFairGAN and the LLM-based GReaT, Curated LLM, and TabGen-ICL, on three standard fairness benchmarks: Adult, Credit scoring, and German credit. They also instantiated the framework with four different large language models, Claude 3.5 Haiku, GPT-4o-mini, Gemini 3 Flash, and Gemini 3 Pro, to test whether the approach generalizes across backbone architectures. Data quality was measured with density, coverage, and distance to closest record; utility with F1-scores of downstream classifiers trained on synthetic data and tested on real data; and fairness with a battery of metrics spanning statistical parity, equalized odds difference, predictive parity, ABROCA, and the Theil Index for individual-level inequality.

The results paint a nuanced but encouraging picture. On the larger and more imbalanced datasets, SENNA variants achieved substantially higher density and coverage than the baselines, and the strongest backbones produced the most locally realistic samples. On utility, SENNA did not dominate every column, which the authors present as a feature rather than a flaw: it remained competitive in F1-score while improving several disparity metrics, offering a balanced trade-off rather than raw performance maximization. Notably, the fairness gains tracked the control mechanisms rather than backbone scale. Stronger models improved fidelity and predictive performance, but the reduction in group and individual disparities came from the disentangled exemplar selection, validation, and balancing stages, suggesting the framework ports well across models.

The study is also candid about its own weaknesses, which lends it credibility. A per-group analysis of the validator revealed that acceptance rates were not uniform: on German credit, female positive-class candidates were accepted at only 0.400 versus 0.756 for the male majority cell, and the smallest post-balancing cells fell far below the nominal 25 percent target. Validation-induced filtering, not just balancing, drives residual imbalance, and the authors propose monitoring acceptance rates as a diagnostic. Quantitative probing showed that disentanglement is approximate rather than complete: on the tiny German credit dataset, a linear probe could still recover label information from the nominally task-independent space, while Credit scoring showed the cleanest separation. The authors also flag that no formal privacy audit, such as a membership-inference attack, has been performed, and that verbatim reproduction of in-context exemplars remains a risk inherent to prompting-based generation.

The broader implications reach beyond one framework. Prior work has shown that repeatedly training on synthetic data can amplify bias through fairness-feedback loops, that fairness measured on synthetic distributions does not always transfer to models trained on them, and that recursive reuse of generated data can cause model collapse. SENNA’s central lesson is that fairness in LLM-based synthesis should be treated as a pipeline-level property, engineered at every stage from representation to post-generation screening, rather than bolted onto a single model. The authors point toward future work on intersectional and multi-valued sensitive attributes, causal fairness guarantees, robustness to distribution shift, and richer serialization beyond templates. For now, the framework, released with full code and configuration, offers practitioners in regulated domains something they have largely lacked: a controllable dial for navigating the unavoidable tension between predictive accuracy and equitable outcomes in synthetic data.

Subject of Research: Fairness-aware synthetic tabular data generation using disentangled semantic encoding and large language models

Article Title: Fair synthetic tabular data generation via disentangled semantic encoding and large language models

Article References: Giang Thi Thu, H., Doan Viet, T., Nguyen Thanh, T., Ta Quoc, T., Le Quy, T., Nguyen Long, G., & Ban, H.-B. (2026). Fair synthetic tabular data generation via disentangled semantic encoding and large language models. Discover Artificial Intelligence, 6(1), Article 1468. https://doi.org/10.1007/s44163-026-02383-w

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02383-w

Keywords: synthetic data, tabular data generation, large language models, algorithmic fairness, disentangled representations, in-context learning, machine learning, group fairness, individual fairness, data privacy, BERT, bias mitigation