AI Virtual Patients Let Medical Students Practice Doctor-Patient Talks

Medical students have long learned the art of the clinical interview by practicing on actors hired to play patients. That gold-standard method is expensive, hard to schedule, and impossible to scale to every student who wants extra practice. Now a team at Universidad de los Andes in Bogotá, working with a researcher at the University of Luxembourg, has built a large language model-based conversational virtual patient that simulates realistic medical interviews and grades the student’s performance automatically, offering a glimpse of how generative AI could reshape clinical communication training.

The system, described in the journal Discover Artificial Intelligence, focuses on the anamnesis, the structured history-taking phase of a medical encounter that can supply up to 80 percent of the information needed for a diagnosis. Students choose from five clinical cases, including type 2 diabetes, chronic obstructive pulmonary disease, and hypothyroidism, and receive no hint of the chief complaint before they begin, mirroring the blank-slate conditions of a genuine first consultation. They can type questions or dictate them through a browser speech-recognition interface, and the simulated patient replies with text accompanied by synthesized speech.

What distinguishes the platform from a simple chatbot is its architecture. The interview is modeled as a directed graph in which each node performs one processing stage, implemented with the LangGraph framework, and a shared state object persists across every turn. Because language models are stateless between invocations and long interviews exceed any context window, the team added a semantic memory layer built with the LangMem library. Clinical facts disclosed by the patient are stored as vector embeddings in a PostgreSQL database extended with the pgvector extension, and retrieved by cosine similarity, so the agent can return to earlier topics using different wording without contradicting itself. Each query retrieves the three most relevant memories, a value tuned empirically to keep response times under two seconds.

Realism comes from a second processing layer that transforms medically accurate replies into the voice of one of five personality profiles: know-it-all, unfriendly, friendly, forgetful elderly, or skeptical. The researchers used probabilistic prompt chaining and few-shot prompting to reshape the output, then synthesized it into audio with dynamically selected voices. The result is an encounter that behaves less like a decision tree and more like a person with moods, inconsistencies, and a memory of what has already been said.

The grading engine is equally ambitious. The team reviewed six validated communication assessment instruments, including the Communication Assessment Tool and the Communication Skills Scale, distilled 88 candidate items down to 24 criteria, and organized them into six categories: overall performance, interest in the patient, empathy, clarity of communication, open communication, and completeness. Each category is scored by a dedicated prompt running in an asynchronous parallel pipeline, producing a 1-to-10 score with a textual justification. Completeness follows a two-phase check in which a generated summary of the interview is first compared against the reference clinical case, then verified against the raw transcript, a correction introduced after physicians complained the tool penalized them for questions they had actually asked.

The validation study involved 21 participants: nine eighth-semester medical students, eleven practising physicians, and one healthcare technology expert, all conducting sessions in Spanish. Students rated the system 82.3 out of 100 on the Chatbot Usability Questionnaire, while the physician and expert cohort gave it 96.5, yielding a pooled usability score of 90.4. Acceptance of the automated feedback averaged 4.50 out of 5 among students and 4.21 among physicians. Interestingly, physicians rated the tool significantly more favorably than students, a gap the authors attribute to several untested hypotheses, including students’ higher baseline expectations shaped by daily exposure to general-purpose AI assistants, and their frustration with missing features such as laboratory results.

The agreement between machine and human judgment was more sobering. In an exploratory comparison against eleven physician ratings of the same interviews, rank agreement was highest for interest in the patient, with a Spearman coefficient of 0.64, and lowest for completeness, at 0.09. No dimension reached statistical significance after correction for six comparisons, and the confidence intervals were wide enough to accommodate anything from negligible to near-perfect agreement. Notably, the automated scores fell below the physician scores in every dimension, suggesting a systematic calibration offset rather than a failure to detect good communication.

The empathy result is particularly revealing. Participants in both cohorts felt the empathy scores were too low, yet empathy actually showed the second-highest rank agreement with human raters and the only confidence interval excluding zero. The system, in other words, ordered students on empathy much as physicians did while assigning harsher absolute scores. The authors point to plausible mechanical explanations: the rubric was derived from instruments covering the entire consultation, including phases where encouragement is expected, and the scoring prompts lacked anchored descriptions of what each point on the scale represents. Both are fixable without changing the underlying model.

The researchers are careful about what the study proves. With a small, single-institution sample, no control group, and remote unsupervised sessions, the findings establish feasibility and acceptability, not pedagogical efficacy. The completeness scorer was even modified partway through data collection, complicating some comparisons. Still, the platform addresses longstanding barriers of cost, logistics, and rigid scripting that have limited traditional standardized patients, and it offers continuous availability with immediate, standardized feedback, qualities especially valuable in resource-constrained settings.

The next step is a controlled trial comparing the system against actor-based training and against a virtual patient without automated feedback, with communication competence measured by blinded human raters before and after the intervention. If the approach holds up, medical students may soon rehearse difficult patients at midnight, receive instant structured critiques, and arrive at their first real ward round having already stumbled through dozens of simulated encounters. The era of practicing medicine on machines before touching patients is no longer hypothetical; it is being scored, one interview at a time.

Subject of Research: LLM-based conversational virtual patients and automated evaluation systems for training medical students in clinical communication and anamnesis

Article Title: Towards clinical communication training with LLM-based conversational virtual patients and automatic evaluation systems

Article References: Bayona, A., Manrique, R., & Grévisse, C. (2026). Towards clinical communication training with LLM-based conversational virtual patients and automatic evaluation systems. Discover Artificial Intelligence, 6(1), Article 1451. https://doi.org/10.1007/s44163-026-02472-w

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02472-w

Keywords: large language models, conversational virtual patient, medical education, clinical communication, anamnesis, automated evaluation, generative agents, semantic memory, LangGraph, usability testing, simulated patients, health communication