Artificial intelligence is quietly moving into hospital wards, and one of its most vulnerable audiences is the elderly. Large language models, the same class of technology behind conversational chatbots, are now embedded in medical devices that help older patients navigate appointments, understand diagnoses and manage chronic conditions. But a striking paradox has emerged in cities like Beijing: the machines are getting smarter while many of the people they serve struggle with basic digital literacy. A new study published in BMC Public Health offers one of the first quantitative portraits of how urban older adults actually perceive the quality of AI-empowered healthcare, and the results carry lessons for hospitals far beyond China.
The research team, led by Xiaoyan Qi and Xuejiao Song of China-Japan Friendship Hospital with corresponding author Xianbo Zuo, surveyed older adults in Beijing who had already used hospital-provided devices built on large language models. Rather than asking whether patients liked the technology, the researchers borrowed a well-established instrument from service marketing science: the SERVQUAL framework. This model, developed decades ago to measure gaps between customer expectations and experiences, breaks perceived service quality into five dimensions: tangibility, meaning the physical and visual aspects of the service; reliability, the ability to perform the promised service dependably; responsiveness, the willingness to help promptly; assurance, the knowledge and courtesy that inspire trust; and empathy, the sense of individualized, caring attention.
Adapting a marketing questionnaire for elderly hospital patients was no trivial task. The team developed a perception-based version of the SERVQUAL instrument and pilot-tested it before deploying it in the field. Out of distributed questionnaires, 298 valid responses were returned and analyzed, a valid response rate of 91.9 percent. The researchers then put the data through a battery of statistical quality checks, including reliability testing with Cronbach’s alpha coefficients ranging from 0.751 to 0.835 across dimensions, confirmatory factor analysis, and tests for common method bias. A Harman single-factor test showed the first factor explained only 28 percent of variance, and a confirmatory one-factor model fit the data poorly while the intended six-factor measurement model fit well, with a comparative fit index of 0.996 and a root mean square error of approximation of just 0.013.
With the measurement instrument validated, the team turned to structural equation modelling, a statistical technique that allows researchers to test relationships between abstract latent constructs, such as perceived assurance, that cannot be directly observed but are inferred from multiple survey items. The analysis revealed that all five SERVQUAL dimensions were significantly and positively associated with older people’s perceived quality of LLM-empowered healthcare services. In other words, every aspect of the service experience, from the look of the device to the warmth of its responses, contributed to how patients judged the overall quality of their AI-mediated care.
Two dimensions stood out with the largest standardized path coefficients: assurance and empathy. This is a finding with real technical and design implications. Assurance in this context reflects whether the AI system conveys competence, credibility and safety, qualities that matter enormously when the user is an older patient making health decisions. Empathy reflects whether the system feels personalized and attentive rather than cold and mechanical. For engineers building medical chatbots, the message is that raw computational accuracy is not enough; the system must communicate trustworthiness and human warmth in ways that older users can perceive.
Yet the researchers were careful not to overstate the hierarchy. They ran constrained comparison tests, forcing each pair of path coefficients to be equal and comparing the resulting model against the unconstrained version. None of these chi-square difference tests reached statistical significance, with all p-values at or above 0.160. Statistically speaking, the study cannot claim that empathy matters more than tangibility or that assurance outranks reliability. What it can claim is that all five dimensions matter, and that no single dimension can be safely neglected when optimizing AI services for older patients.
The structural model explained 57.0 percent of the variance in perceived service quality, a substantial share for research on human perceptions of technology. The remaining variance presumably reflects factors outside the five-dimension framework, from individual digital literacy to health status to prior experience with technology. The authors also reported a supplementary multiple regression on observed composite scores, which yielded a lower R-squared of 0.386, a difference they attribute to the level of analysis, since latent variable models account for measurement error that observed-score regressions do not.
The study’s broader significance lies in what it says about the digital divide in healthcare. Older adults often face a practical paradox between device intelligence and low digital literacy: the more sophisticated the technology, the wider the gap for those who did not grow up with smartphones and voice assistants. By demonstrating that a perception-based SERVQUAL approach can be applied to AI-mediated geriatric care, the researchers provide a reusable framework for measuring whether these systems actually serve their intended users. The findings offer preliminary, perception-based evidence to inform age-friendly technical optimization of LLM-based services, hybrid online-offline service models and AI governance policies.
The practical improvement pathways suggested by the work point in several directions at once. Hospitals deploying large language model devices may need to invest in the tangible cues that signal quality to elderly users, ensure the systems respond reliably and promptly, build in visible safeguards that create assurance, and design conversational interfaces that convey empathy rather than bureaucratic detachment. At the same time, the authors emphasize that their evidence is perception-based and cross-sectional, drawn from a single city and a single point in time, so causal claims and generalizations require further research.
As large language models spread through healthcare systems worldwide, studies like this one serve as an early warning and a guide. The technology’s success will not be decided by benchmark scores alone but by whether an eighty-year-old patient at a hospital kiosk feels understood, safe and respected by the machine in front of them. Measuring that feeling rigorously, dimension by dimension, may prove just as important to the future of AI in medicine as any algorithmic breakthrough.
Subject of Research: Quality of large language model-empowered geriatric healthcare services as perceived by urban older adults
Article Title: Factors associated with the quality of urban geriatric healthcare services empowered by large language models and improvement pathways: a theoretical framework and cross-sectional study
Article References: Qi, X., Song, X., Xu, J., Chen, L., cui, Y., & Zuo, X. (2026). Factors associated with the quality of urban geriatric healthcare services empowered by large language models and improvement pathways: a theoretical framework and cross-sectional study. BMC Public Health. https://doi.org/10.1186/s12889-026-29712-z
Image Credits: AI Generated
DOI: 10.1186/s12889-026-29712-z
Keywords: large language models, geriatric care, SERVQUAL, healthcare service quality, older adults, digital divide, structural equation modelling, artificial intelligence, patient perception, Beijing, AI governance, digital health
News Source: Ophelia Keating. (October 10, 2026). What Older Patients Really Want From AI-Powered Hospital Care. Scienmag.
