AI Agents Debate to Uncover Users’ Core Values for Better Recommendations

Recommender systems have become the invisible engine of the modern internet, deciding which news articles, products, and films appear in front of billions of people every day. Yet according to a new study published in the journal Data Science and Engineering, these systems have a fundamental blind spot: they excel at tracking what users click in the moment but remain largely blind to who those users actually are at their core.

The problem, the researchers explain, lies in the nature of the data on which most recommenders are trained. User behavior is dynamic, sparse, and noisy. A person may click on a sensational news article about a violent crime out of momentary curiosity, even though their underlying personal value is benevolence. A conventional system interprets that click as a positive signal and keeps serving similar content, gradually degrading the user’s experience. What distinguishes a passing interest from a genuine preference, the authors contend, is the concept of user values: stable, intrinsic factors that shape long-term decision-making, such as which items people purchase or which content they consume over months and years.

The obstacle is that user values are almost never available in the data. In psychology and related fields, they are typically measured through surveys and questionnaires administered before users engage with a service. That approach is expensive, time-consuming, and requires trained researchers to design the instruments. Engagement rates with lengthy surveys are usually low, and self-reported answers may not reliably reflect true preferences. The team therefore set out to do something that had remained underexplored: infer user values automatically from nothing more than a user’s historical interaction record, using the language understanding and zero-shot reasoning abilities of large language models.

Directly asking a large language model to read a user’s entire interaction history and declare their values, however, runs into two well-documented walls. The first is context length. Popular open models such as Llama-3-8B accept only about 4,096 tokens of input, and GPT-3 roughly 2,048. Interacted items carry extensive metadata, including titles, attribute descriptions, and main content, so a realistic history quickly exceeds what a model can process in one pass. The second wall is hallucination. Prior studies have shown that large language models frequently generate outputs that are unfaithful or nonsensical relative to their source material, and the problem becomes more pronounced on complex inference tasks, of which extracting a person’s fundamental values from click streams is a prime example.

The researchers’ answer is a framework called ZOOM, a zero-shot, multi-LLM collaborative system for user value mining. To give the extraction a principled structure, ZOOM adopts Schwartz’s Theory of Basic Values, a framework from cross-cultural psychology that defines ten universal basic values, organized into four higher-level groups: Openness to Change, Conservation, Self-Enhancement, and Self-Transcendence. The ten values are Achievement, Power, Hedonism, Stimulation, Self-Direction, Security, Conformity, Tradition, Universalism, and Benevolence, with 58 fine-grained value items beneath them. The theory’s clarity, cross-cultural adaptability, and transparency make it a useful scaffold for machine-generated judgments, and it even supplies internal consistency checks: because Self-Direction belongs to Openness to Change while Tradition belongs to Conservation, a value set containing both would be flagged as inappropriate.

ZOOM’s architecture is a pipeline of specialized agents. First, a summarization stage prompts a language model to condense the content of each interacted item while preserving its core meaning, allowing far more of the user’s history to fit within the token limit. Then two evaluator agents process the summarized history along different reasoning paths: one reads the items in their original chronological order, the other in a randomized order, a deliberate hedge against the position bias that studies show can distort LLM judgments. Each evaluator generates candidate value sets using four decoding strategies: beam search, temperature sampling, top-k sampling, and nucleus sampling, producing hypotheses that range from high-likelihood and low-variance to deliberately exploratory.

From there, the framework shifts from generation to scrutiny. Each evaluator is paired with a supervisor agent that critically reviews the candidates and selects the value set it judges most faithful to the interaction history. The two supervisors, built on different backbone models such as Llama-3-8B and Gemma-2-9B to maximize diversity, then engage in a debate lasting up to three rounds, critiquing each other’s selections and revising their own positions in light of the shared debate history. If they converge, the consensus becomes the user’s final value set. If they exhaust the rounds without agreement, a final supervisor reviews the entire debate trace and makes the binding decision. The design draws on the principle that multiple correct reasoning paths should lead to the same conclusion, and on prior work showing that multi-agent debate helps filter out inconsistent, hallucinated claims while preserving facts that achieve stable consensus.

Mining values is only half the problem; the other half is injecting them into a working recommender without breaking it. The authors tested two fusion strategies. Direct concatenation simply appends the user value embedding to the model’s preference embedding, but this can backfire when the value vector dominates through dimensional imbalance or when the two representations live in misaligned semantic spaces. Their preferred alternative is a contrastive-learning-based auxiliary task: the user’s preference embedding and its matching value embedding are treated as a positive pair, while the N value embeddings least similar to the user’s preferences serve as negatives, and the training objective pulls the matched pair together while pushing unrelated values apart. The value embeddings themselves come from a pre-trained language model and are frozen during recommendation training, keeping value representations consistent across users.

The experimental evidence comes from two widely used datasets, PENS, a Microsoft news dataset with fifteen categories, and MovieLens-1M, with eighteen movie genres, evaluated on two state-of-the-art language-model-based recommenders, MoRec and EasyRec, using leave-one-out testing and the HR@K and NDCG@K metrics computed over all unseen items. Integrating values helped in nearly every configuration, and the contrastive approach consistently beat both the original models and direct concatenation across all metrics. The largest gains appeared with EasyRec on MovieLens-1M, where direct concatenation lifted HR@10 by 24.53 percent and NDCG@20 by 35.95 percent, with improvements exceeding 30 percent across all metrics for that pairing. The authors attribute this to EasyRec’s text-based collaborative filtering, whose user embeddings are already derived from textual profiles and therefore align more naturally with text-derived value embeddings, whereas MoRec’s sequential modeling makes integration harder. Gains were smaller on the sparser PENS dataset, where fewer interactions per user limit both value mining and recommendation modeling.

The framework’s own accuracy was checked against human annotators who labeled ground-truth values for randomly selected users following Schwartz’s theory, with cross-validation among evaluators. On 50 users per dataset, ZOOM achieved F1 scores of 0.6813 on PENS and 0.7282 on MovieLens-1M, more than six times higher than a naive LLM baseline. An ablation study underscored how much each component matters: removing summarization collapsed the F1 score to 0.2068 on PENS and 0.3200 on MovieLens-1M, because the model could then fit only two or three items and formed a distorted picture of each user, while removing the debate stage more than halved performance to 0.2680 and 0.3600 respectively. A case study of one PENS user illustrated the difference concretely: a general LLM labeled the user’s value as Achievement, which the authors judged inappropriate given the full history, whereas ZOOM identified Benevolence and Universalism, better supported by the content the user actually read. Hyperparameter analysis showed the best contrastive loss weight at 0.01 across settings, while the optimal number of negative samples varied by dataset, from 3 to 5 on PENS, where value distributions are skewed toward a few dominant values, to 7 to 9 on the more diverse MovieLens-1M. The authors note that the mining process is a one-time offline computation, upper-bounded at 17 LLM calls per user, so it does not slow real-time recommendation, and they flag validating LLM-generated outputs as an open challenge for future work.

Subject of Research: Automatic mining of user values from interaction histories using multi-agent large language models to improve recommender systems

Article Title: Multi-agents Based User Values Mining for Recommendation

Article References: Chen, L., Yuan, W., Chen, T., Zhao, X., Nguyen, Q. V. H., & Yin, H. (2026). Multi-agents Based User Values Mining for Recommendation. Data Science and Engineering. https://doi.org/10.1007/s41019-026-00349-7

Image Credits: AI Generated

DOI: 10.1007/s41019-026-00349-7

Keywords: recommender systems, large language models, multi-agent frameworks, user values, Schwartz theory of basic values, hallucination mitigation, contrastive learning, zero-shot inference, PENS dataset, MovieLens-1M, text summarization, personalization