Understanding how choice overload in ChatGPT recommendations impacts decision-making

Over the past few years, the field of artificial intelligence (AI) has witnessed numerous breakthroughs. One such remarkable milestone was the development and adoption of chatbots and conversational agents based on large language models, including ChatGPT. These systems can engage in realistic, human-like conversations with users and help them in many ways, such as by curating information, generating recommendations, or assisting in complex tasks. Interestingly, owing to their pre-training on large amounts of data, chatbots like ChatGPT are capable of generating highly personalized recommendations, considering factors like user’s interests, browsing history, and their preferences.

With these advanced capabilities, it is likely that chatbots will soon take the cyberworld by storm and make their way into daily life decision-making, across industries such as retail, manufacturing, finance, tourism, and customer service. Thus, understanding how consumers perceive their responses is just as important as the responses themselves.

Against this backdrop, a research team led by Professor Changju Kim from College of Business Administration, Ritsumeikan University, Japan, recently conducted a detailed study to investigate how the problem of choice overload impacts ChatGPT users and its impact on their decision-making process. This study was made available online on July 13, 2023, and will be published in Volume 75 of the Journal of Retailing and Consumer Services in November 2023.

The ‘choice overload’ phenomenon occurs when a person is overwhelmed by the number of options while making decisions. When consumers experience choice overload, they often find it challenging to decide, fearing that they might make the wrong one. In many cases, this can result in decreased satisfaction with the chosen option, which is counterproductive in most contexts.

But do users experience a similar choice overload when a chatbot like ChatGPT is the one providing multiple options? Prof. Kim and his team attempted to find an answer to this question. Elaborating on this further, Prof. Kim says, “Despite their advantages, chatbots like ChatGPT also have certain limitations, such as those associated with privacy, information transparency, and incorrect information. Moreover, little is known about how ChatGPT influences consumer decision-making. To shed light on this aspect, we decided to analyze consumers’ responses to a relatively large number of options suggested by ChatGPT.”

In traditional decision-making, about 24 to 30 options are enough to induce choice overload. However, the research team theorized that the negative effects associated with such a large number of options would decrease if they were generated by ChatGPT, given its ability to provide highly personalized and accurate recommendations.

The researchers tested their hypothesis by conducting five independent studies between February and March 2023 to analyze consumers’ responses to the recommendation options provided by ChatGPT. In the first two studies, participants received song recommendations and were asked to assess their perceived satisfaction, accuracy, and intent to purchase. In the other three studies, participants received suggestions on places to visit during a hypothetical trip to Kyoto, Japan. These studies not only focused on the number of options but also considered the source of the suggestions. Participants were then asked to respond to these recommendations by rating their satisfaction, perceived accuracy, and intention to visit the recommended place.

The results of these five studies provided interesting insights into chatbot-assisted decision-making. The researchers found that participants preferred a large number of recommendation options, such as 60 or 70, from ChatGPT. Their satisfaction and intent to purchase increased with the number of options because they perceived the information provided by the chatbot to be accurate. Moreover, participants preferred receiving many suggestions from ChatGPT, compared to those from a human or an online travel agent. These findings suggest that the nature of recommendation agent greatly influences the number of options preferred by a consumer.

These findings can have important implications for real-world applications. For instance, businesses can leverage the perceived accuracy of the information provided by ChatGPT and provide consumers with multiple options, without the fear of negatively impacting their decision-making. In turn, this would lead to people making better decisions more conveniently, eliminating the need to perform complex searches. Moreover, developers can design highly customized and user-friendly recommendation systems that match the needs and preferences of the consumers. This would be invaluable in industries such as tourism and online shopping, making it easier for consumers to make decisions.

Satisfied with their results and with eyes on the future, Prof. Kim says, “ChatGPT represents a significant advancement in the field of recommendation systems, as it recommends products, services, places, people, or any other solutions that align better with the needs and preferences of consumers. Our findings underscore the need for a better understanding and application of AI-generated recommendations in real-world contexts as well as information accuracy and personalization of these recommendations.”

Let us hope that we find more ways to make our lives easier with AI!

***

Reference

DOI: https://doi.org/10.1016/j.jretconser.2023.103494

About Ritsumeikan University, Japan

Ritsumeikan University is one of the most prestigious private universities in Japan. Its main campus is in Kyoto, where inspiring settings await researchers. With an unwavering objective to generate social symbiotic values and emergent talents, it aims to emerge as a next-generation research university. It will enhance researcher potential by providing support best suited to the needs of young and leading researchers, according to their career stage. Ritsumeikan University also endeavors to build a global research network as a “knowledge node” and disseminate achievements internationally, thereby contributing to the resolution of social/humanistic issues through interdisciplinary research and social implementation.

Website: http://en.ritsumei.ac.jp/

About Professor Changju Kim from Ritsumeikan University, Japan

Changju Kim is a Professor at the College of Business Administration at Ritsumeikan University since April 2019. Prior to this, he served as an Assistant Professor (April 2010 to March 2013) and Associate Professor (April 2013 to March 2019) at Ritsumeikan University. Prof. Kim has received Master’s and Ph.D. degrees from Cardiff University, UK, and Osaka City University, Japan, in 2005 and 2010, respectively. His main areas of study are retailing, sales management, political consumption, and cross-cultural marketing. He has published around 30 research papers, with over 200 citations. He is a member of The Japan Society of Marketing and Distribution, Korea Distribution Association, and Global Alliance of Marketing and Management Associations.