ChatGPT-generated answers to patient questions about total knee arthroplasty received near-perfect ratings from three independent arthroplasty surgeons in a new cross-sectional evaluation published in Archives of Orthopaedic and Trauma Surgery. Across 70 patient questions, 50 covering conventional knee replacement care and 20 focused on fast-track recovery pathways, the evaluated responses scored a mean of 4.99 out of 5 for accuracy, 4.98 for completeness, 4.98 for safety, and 5.00 for readability on 5-point scales. Every one of the 20 fast-track responses received a score of 5 from each of the three raters in all four domains, producing 240 out of 240 maximum ratings. Yet the authors themselves caution that these striking numbers come with important caveats, and that clinician review of artificial intelligence-generated medical advice remains essential.
The study addressed a growing practical question in joint replacement care. Fast-track and Enhanced Recovery After Surgery pathways now dominate contemporary arthroplasty practice, emphasizing patient education, multimodal opioid-sparing analgesia, early mobilization, avoidance of unnecessary drains and urinary catheters, and discharge based on functional criteria rather than fixed hospital stays. As length of stay shrinks, patients and caregivers have less time to ask questions during hospitalization, which raises the stakes for accurate and accessible preoperative and postoperative information. At the same time, patients increasingly turn to large language models such as ChatGPT, which differ from traditional search engines by generating individualized, conversational responses to practical concerns about wound care, pain, mobilization, complications, and return to activity.
To build the question bank, the researchers started with 91 patient questions drawn from outpatient arthroplasty consultations and publicly available patient education materials. A modified Delphi panel of three high-volume arthroplasty surgeons, one physiotherapist, and one orthopaedic nurse then scored each question on frequency in clinical practice, clinical relevance, and importance for patient safety, and the 50 highest-ranked conventional questions were selected. Separately, 20 fast-track questions were written by clinicians experienced in fast-track total knee arthroplasty who were not among the subsequent raters, covering early mobilization, early discharge, multimodal analgesia, selective use of drains and catheters, postoperative self-management, thromboprophylaxis, and safety after discharge.
Responses were generated in Polish in June 2026 using GPT-6 Sol through ChatGPT, with a standardized instruction asking the model to answer in patient-friendly language consistent with current medical literature. Each question was entered into a newly initiated conversation to minimize contextual influence, and all responses were recorded verbatim without any post-generation editing or correction. Three high-volume arthroplasty surgeons then independently evaluated every response in four domains: accuracy, completeness, safety, and readability, each scored from 1 to 5. The raters were blinded to one another’s ratings and were not told the subgroup labels, although the authors note that fast-track topics could readily reveal the group, so effective category masking cannot be assumed.
The quantitative results were remarkable in their uniformity. Recalculation from all 840 individual domain ratings confirmed the subgroup statistics, and the 20 fast-track responses each received unanimous maximum scores, with the absence of disagreement verified as genuine in the supplied assessment sheet rather than an artifact of averaging. Exact three-rater agreement across all 70 responses reached 68 of 70 for accuracy, 67 of 70 for completeness, 68 of 70 for safety, and 69 of 70 for readability, with every disagreement occurring among conventional questions. Paradoxically, despite this near-total concordance, single-rating intraclass correlation coefficients were near zero to low, a statistical consequence of nearly every score being 5 and the sparse non-5 ratings being concentrated in only three conventional questions. The fast-track intraclass correlations could not even be estimated, because both between-question and within-question variance were zero.
Safety flags were rare but not absent. Across the 210 individual expert assessments, three assessments, or 1.4 percent, indicated that a response required correction by a specialist, involving two unique responses, and one assessment, or 0.5 percent, identified a response as potentially unsafe, corresponding to one unique response. All flagged responses belonged to the conventional question group; no fast-track response was flagged. The sole potentially unsafe flag concerned a question about prophylactic antibiotics before dental treatment after knee replacement. The answer opened by saying routine prophylaxis is generally unnecessary and then described individual exceptions. One rater marked it for correction and as potentially unsafe, preferring prophylaxis for reassurance and implant protection, while the other two scored its safety 5 out of 5 and did not flag it.
In a subsequent author discussion, the group agreed the wording could be misunderstood, since a reader might take the opening general advice as applying even when individual risk warrants a separate decision. The authors emphasize this is an interpretation of the wording rather than a documented factual error, and note that the response’s general recommendation aligns with American Dental Association guidance and the limited-strength American Academy of Orthopaedic Surgeons option that routine prophylaxis may not reduce subsequent prosthetic joint infection risk. Importantly, there was no independent, prespecified adjudication of this flag, and the authors caution that a single flag alone should not be interpreted as a confirmed harmful answer.
The study’s limitations are central to interpreting its conclusions. The conventional and fast-track question sets were assembled through entirely different processes and did not share a sampling or selection framework, so their wording, scope, and difficulty could have affected the ratings. Scores clustered at the scale ceiling, and the between-group mean differences of only 0.01 to 0.03 points are small relative to the conventional group’s response-level standard deviations of 0.05 to 0.13. The exploratory Mann-Whitney U comparisons, whose p-values the authors independently reproduced using SciPy, were explicitly not designed as tests of equivalence, and nonsignificant results do not prove equivalence. Other limitations include a single generation period, Polish-language responses only, surgeon-only raters, static standardized questions, and no measure of patient comprehension or clinical outcomes.
The findings align with a growing body of orthopaedic literature. Previous studies have found that ChatGPT can answer common hip and knee arthroplasty questions with adequate accuracy and comprehensiveness while cautioning against its use as a sole information source, and recent work reported high surgeon and patient satisfaction with ChatGPT responses to real knee replacement questions. Related research on preoperative counselling and informed consent similarly found generally high-quality information alongside calls for critical evaluation and verification against established medical sources. This study extends that literature by specifically addressing fast-track concepts, which can differ from traditional patient expectations of prolonged bed rest, longer hospitalization, or more passive recovery, and which require explanations that avoid implying a single pathway applies to every patient.
The authors conclude that experts rated responses to both conventional and fast-track questions highly in this sample, but that the different question-selection procedures and limited score variation prevent any conclusion of equivalent performance between the sets. They recommend that future comparisons apply the same independent question-selection procedure to both sets, mask group labels where feasible, include varied question difficulty, and prespecify an equivalence margin and adequate sample size if equivalence is the research question. Practically, they suggest artificial intelligence-generated content may be most useful when integrated into supervised patient education workflows, such as preoperative information platforms, postoperative FAQ systems, or clinician-approved discharge resources. Prospective studies are still needed to determine whether AI-supported education can improve patient satisfaction, reduce unnecessary postoperative contacts, or enhance adherence to fast-track recovery recommendations. Until then, the message is measured: the evaluated answers were excellent by expert judgment, but clinical review remains necessary for safety-critical and protocol-dependent recommendations.
Subject of Research: Expert evaluation of ChatGPT-generated responses to patient questions about conventional and fast-track total knee arthroplasty
Article Title: Expert evaluation of ChatGPT-generated responses to patient questions in conventional and fast-track total knee arthroplasty
Article References: Maciag, B., Jegierski, D., Frankiewicz, A., Tarkowski, P., Żarnovsky, K., Mietlicki, F., & Stolarczyk, A. (2026). Expert evaluation of ChatGPT-generated responses to patient questions in conventional and fast-track total knee arthroplasty. Archives of Orthopaedic and Trauma Surgery, 146(1), Article 370. https://doi.org/10.1007/s00402-026-06534-7
Image Credits: AI Generated
DOI: 10.1007/s00402-026-06534-7
Keywords: ChatGPT, total knee arthroplasty, fast-track arthroplasty, patient education, large language models, ERAS, expert evaluation, orthopaedic surgery, artificial intelligence, patient safety, readability, surgeon ratings
News Source: Ophelia Keating. (October 29, 2026). Surgeons Give ChatGPT Near-Perfect Marks for Knee Replacement Patient Answers. Scienmag.
