AI Agents Replicate Reddit Patterns but Distort Toxicity Levels

Published in EPJ Data Science, the study provides a rigorous, replicated validation of LLM-agent social simulations by comparing 30 independent 30-day simulations against 30 matched, non-overlapping windows of real data from Voat’s v/technology subverse. The work addresses a critical gap in generative agent-based modeling, where previous studies often relied on subjective believability assessments or single-run demonstrations without quantitative empirical comparison.

The study utilized the Y Social platform, a client-server digital twin of a Reddit-like forum, to host stateless agents powered by the Dolphin Mistral 24B model. By using an uncensored base model, the researchers aimed to preserve the potential for disagreement and toxic speech that standard aligned models often suppress. The agents were assigned personas including education level, political leaning, and toxicity propensity, but they operated without long-term memory. Continuity was maintained solely through server-side state, such as thread history and user interests, passed into each prompt during short, stateless micro-dialogues.

To ensure the simulation was grounded in reality, the team calibrated population size, growth rates, and posting likelihoods using the MADOC dataset, which contains standardized data from Voat. The calibration process involved analyzing 10 non-overlapping 30-day windows to determine daily activity metrics, user dynamics, and thread characteristics. This revealed that v/technology is a small, volatile forum with approximately 576 unique users per 30-day window and high churn rates, indicating less persistent engagement. The simulation emulated these conditions by initializing 50 agents and updating the population daily according to calibrated growth and churn rates.

The validation framework assessed operational validity across five dimensions: activity patterns, network structure, toxicity, topical coverage, and stylistic convergence. Operational validity requires that simulated aggregate distributions resemble those of the reference community at comparable scales, without one-to-one reproduction of specific users or texts. The researchers used 99% t-distribution confidence intervals from the 30 simulation runs to compare metrics against the empirical Voat benchmarks. This approach allowed for statistical inference and the identification of systematic divergences linked to specific architectural choices in the simulation design.

Results showed that the simulation successfully matched the scale and activity rhythms of the real forum. Root-post volume, unique user counts, and daily active users overlapped with the Voat benchmarks, indicating that the agents produced a plausible community size and engagement level. The distribution of root posts per user was strongly right-skewed in both datasets, but the simulation was more tightly concentrated around low-activity modes, whereas Voat retained more mass in the extreme participation tail.

Network analysis revealed that both simulated and empirical networks exhibited core-periphery structure, but the simulated cores were larger and more diffuse. The average degree in the simulation was 2.72, compared to 2.31 in Voat, and the clustering coefficient was 4.6 times higher in the simulation. This indicates that the simulated network was denser and more interconnected than the real one. Repeated interactions were also less frequent in the simulation, with only 5.90% of edges representing repeated exchanges, compared to 8.70% in Voat. These differences suggest that the stateless design and simplified feedback rules dampen the formation of strong, persistent hubs seen in real communities.

Toxicity analysis uncovered a significant layer-dependent mismatch. While overall mean toxicity was higher in the simulation (0.143) than in Voat (0.119), the distribution was skewed. Simulated root posts were substantially more toxic than real submissions, with a mean toxicity of 0.116 compared to 0.0307 in Voat. This misallocation suggests that persona calibration and opening-post prompts encouraged overly confrontational starts, while the reply process underproduced the harsher tone typically found in real comment sections. A depth-shuffle permutation test indicated that this pattern was driven by the post-comment split rather than within-thread escalation.

Topical coverage was near-complete, with 99.6% of simulation topics having at least one match in the Voat corpus. Core technology themes such as privacy, AI, and electric vehicles showed strong alignment, with cosine similarities between topic centroids ranging from 0.69 to 0.82. This supports the content validity of the simulation, showing that the agents’ interests and the fixed link catalog successfully anchored discussions to relevant technological topics. Embedding similarity analysis further confirmed that simulated posts aligned moderately well with real Voat posts, although comments showed slightly lower alignment, reflecting more varied commentary styles in the real data.

Stylistic convergence was measured using convergence entropy, which quantifies short-range alignment in reply chains. The simulation exhibited lower entropy than the GPT-4o mini baseline, indicating greater stylistic convergence. However, this convergence was a direct consequence of the threaded discussion structure and the configured context window, not stylistic persona adaptation. Entropy rose monotonically with turn distance, plateauing once prior turns dropped out of the context window. This finding confirms that the agents did not exhibit persistent stylistic learning or identity alignment, but rather local accommodation mediated by the architectural constraints of the platform.

The study highlights that while LLM agents can reproduce familiar online regularities, systematic divergences point to concrete directions for improvement. The stateless design, which lacks long-term memory, limits the simulation’s ability to capture long-range conversational carryover and identity formation. The researchers note that the current results represent evidence of validity in a tractable small-community regime, and that larger or more heterogeneous communities would likely amplify the observed mismatches in hub consolidation and repeated interactions. Future work will focus on introducing agent memory and testing alternative feed variants to better capture the dynamics of real online discourse.

The implications of this research extend beyond technical simulation, offering a baseline for understanding how artificial users might interact in mixed human-machine environments. As online platforms begin to integrate AI agents, understanding what these agents alone can produce provides a critical reference for studying human-machine interactions. The study’s operational-validation framework offers a reproducible approach for assessing the fidelity of future LLM-based social simulations, ensuring that claims of realism are grounded in quantitative comparison with empirical data rather than subjective judgment.

The authors emphasize that the use of an uncensored model was a realism choice to capture contentious discourse, not a policy statement. They caution that the findings should be interpreted as structural resemblance at the right scale, not literal replication of individual user behavior. The transparency of the workflow, including the open-access availability of code and data, allows for cumulative empirical research on online social network dynamics. This work sets a new standard for validating generative agent-based models, moving the field from anecdotal demonstrations to rigorous, falsifiable scientific inquiry.

The simulation architecture relies on a specific mechanism to generate agent behavior: a stochastic action menu that enforces unequal participation. Each agent receives a per-round action budget sampled from a truncated Zipf distribution, ensuring that most agents remain minimally active while a small minority contribute disproportionately. This design choice mirrors the participation inequality observed in real online communities. During each turn, agents select from a limited set of options, including posting, commenting, reading, searching, or sharing links. The platform’s feed is non-personalized and ranked by popularity, with ties broken by recency, within a fixed visibility window of 96 rounds. This setup allows the simulation to test whether compressed cultural knowledge, combined with realistic platform constraints, can produce emergent social patterns without agents optimizing for specific rewards or visibility.

Agent personas were constructed to reflect the ideological profile of the v/technology subverse, which is noted for its right-of-center skew. Political leaning was the only benchmark-informed attribute, with agents sampled from four segments corresponding to Pew’s 2021 typology. Other persona dimensions, such as education level and age, were sampled broadly for face validity rather than demographic representativeness. Each agent was also assigned a toxicity propensity level, with the majority labeled as having no toxicity. This stratification allowed the researchers to verify that the toxicity ladder mechanism produced the intended behavioral gradient, confirming that toxicity variation followed from designed persona traits rather than being an artifact of the base model’s tendencies.

The validation process included a sensitivity analysis using a separate 10-day setup to test the robustness of the toxicity findings. In this variant, a neutral-persona configuration reduced mean toxicity by 46.7% and the share of items above a toxicity threshold of 0.5 by 51.5%. In contrast, a no-politics variant produced only modest shifts with overlapping confidence intervals. This finding suggests that the observed toxicity mismatches in the main 30-day benchmark are tunable through calibration adjustments rather than being fundamental limitations of the LLM approach.

The study’s scope is intentionally limited to a small, shallow-threaded community, which serves as a favorable first test for stateless agents. The researchers caution that larger or more heterogeneous communities would likely amplify the observed mismatches, particularly in hub consolidation and long-range conversational carryover. The stateless design, which lacks long-term memory, improves the interpretability of short-range effects but limits the simulation’s ability to represent long-range phenomena such as identity formation or persistent stylistic alignment. Future extensions will focus on introducing lightweight, per-thread memory and testing alternative feed variants, such as pure recency or controversial ranking, to probe how different recommendation strategies affect core structure and toxicity dynamics.

Subject of Research: Validation of LLM-agent social simulations against empirical Voat forum data

Article Title: Towards operational validation of LLM-agent social simulations: a replicated study of a Reddit-like technology forum

Article References: Tomašević, A., Cvetković, D., Major, S., Maletić, S., Anđelković, M., Vranić, A., Stupovski, B., Vudragović, D., Bogojević, A., & Mitrović Dankulov, M. (2026). Towards operational validation of LLM-agent social simulations: a replicated study of a Reddit-like technology forum. EPJ Data Science, 15(1), Article 72. https://doi.org/10.1140/epjds/s13688-026-00674-x

Image Credits: AI Generated

DOI: 10.1140/epjds/s13688-026-00674-x

Keywords: LLM agents, social simulation, operational validity, Voat, network analysis, toxicity, agent-based modeling, digital twin, Y Social, Dolphin Mistral, Towards, operational