Recent research from the University of California San Diego has unveiled a striking phenomenon: chatbots, particularly those powered by large language models (LLMs), significantly influence consumer behavior by altering the sentiment in product reviews. The findings indicate that potential buyers are 32% more likely to purchase a product after engaging with a review summary created by a chatbot compared to reading the original human-written review. This enhancement in persuasion occurs due to an inherent bias—specifically, a tendency towards favorable framing—that chatbots introduce when summarizing text.
In this groundbreaking study, the researchers quantitatively measured the effects of cognitive biases introduced by LLMs on decision-making. They discovered that LLM-generated summaries reframe the original sentiments of reviews in 26.5% of instances. Moreover, the study revealed a staggering figure: LLMs hallucinated — or provided inaccurate information — approximately 60% of the time when respondents questioned them about news stories, especially when these stories deviated from the training data used in training the models. The researchers characterized this tendency toward generating misleading information as a significant limitation, pointing out the difficulty these models face in reliably distilling fact from fiction.
So, how do these biases seep into the outputs of LLMs? The models often lean heavily on the initial segments of the text they summarize, neglecting to capture essential nuances that may emerge later in the review. This over-reliance on the early context, along with diminished performance when challenged with information beyond their training set, cultivates an environment ripe for biased summarization.
In order to deepen the understanding of the impact these biases have on consumer decisions, researchers implemented a study involving 70 participants who were presented with either the original review or summaries generated by LLMs for various products such as headsets, headlamps, and radios. Astonishingly, the results showed that 84% of participants who read the LLM-generated summaries expressed their intention to purchase the product, in stark contrast to only 52% of those who read the original human reviews. This striking difference underscores the profound influence that the framing of information can have on purchasing decisions.
The research team was surprised by the extent of the effect that the summarization had in their low-stakes experimental context. Specifically, Abeer Alessa, the lead author of the study and a master’s student in computer science, acknowledged the potential for an even more significant impact in high-stakes scenarios where major decisions are at play. This revelation raises questions about the ethical implications of using LLMs in contexts where consumer choices can have far-reaching effects.
In search of solutions to mitigate the issues identified, the researchers explored 18 distinct methods to address cognitive biases and hallucinations. They found that while some mitigation strategies proved effective for particular models in specific situations, there was no singular approach that worked universally across all LLMs. Furthermore, some mitigation techniques appeared to introduce new challenges, potentially compromising LLM performance in other critical areas.
Julian McAuley, a senior author of the paper and a professor of computer science at UC San Diego, emphasized the nuanced nature of bias and hallucination in LLMs. He explained that effectively fixing the issues tied to bias and hallucinations is complicated, requiring a contextualized approach rather than blanket solutions. These challenges highlight the intricate interplay between AI-generated content and human understanding.
The study assessed various models, including small open-source configurations like Phi-3-mini-4k-Instruct, Llama-3.2-3B-Instruct, and Qwen3-4B-Instruct. They also evaluated a medium-sized model, Llama-3-8B-Instruct, as well as larger models like Gemma-3-27B-IT and a proprietary model, GPT-3.5-turbo. This diverse array of models provided a fertile ground for examining the effects of LLMs on the generation of potentially biased and misleading content.
The researchers posit that their findings represent a crucial leap toward analyzing and addressing the content alterations induced by LLMs on human decision-making. By shedding light on these biases, the research aims to foster a deeper understanding of how LLMs can influence media, education, and public policy. The study emphasizes the need for ongoing discourse and research to navigate the complexities of AI-generated content and its ramifications on society.
In December 2025, the researchers presented their work at the esteemed International Joint Conference on Natural Language Processing and the Asia-Pacific Chapter of the Association for Computational Linguistics, signaling ongoing interest and inquiry in the field of artificial intelligence. This research holds promise not only for advancing the understanding of language models but also for guiding the ethical application of AI in various domains.
As the conversation around AI ethics continues to grow, the implications of such research become ever more pressing. The findings from UC San Diego emphasize that while LLMs promise efficiency and versatility in content creation, they also carry the potential for unwanted biases that could skew user perceptions and decisions. To harness the power of these technologies responsibly, it is imperative for developers and users alike to be mindful of the subtleties that influence how information is perceived and acted upon.
Given the increasing prevalence of AI in everyday decision-making contexts, the research serves as a vital reminder of the need for caution. As LLMs are integrated into more facets of daily life, from shopping to information dissemination, ensuring the integrity of the content they generate must remain a priority. A commitment to transparency, accountability, and ethical guidelines in deploying LLMs can help mitigate unintended biases and safeguard against their potential consequences.
Subject of Research: People
Article Title: Chatbots’ Bias Makes Consumers More Likely to Buy Products Suggests New Study
News Publication Date: October 2023
Web References:
References:
Image Credits: David Baillot/University of California San Diego
Keywords: Cognitive Bias, Large Language Models, Consumer Behavior, Artificial Intelligence, Product Reviews, Decision Making, Mitigation Strategies, Research Study.
Tags: accuracy challenges of chatbot informationchatbot bias and user perceptioncognitive biases in decision-makingeffects of framing on consumer choiceshallucination issues in AI responsesimpact of large language models on consumer behaviorimplications of chatbot design on user trustinfluence of AI-generated content on purchaseslimitations of language models in summarizationpersuasive technology in product reviewsresearch on AI ethics and biasessentiment alteration in product reviews
