Retrieval-Augmented Language Model Detects Severe Deterioration in Mid-Acuity Emergency Patients

A retrieval-augmented large language model significantly improved the detection of severe early clinical deterioration among mid-acuity emergency department patients compared with a prompt-only baseline, according to a retrospective case-control study published in BMC Emergency Medicine. Mid-acuity attendances in emergency settings represent a clinically diverse group where only a small minority experience rapid, life-threatening decline. While identifying these vulnerable individuals early is vital, widespread escalation of mid-acuity patients would impose an impractical review burden on clinical staff, creating a need for targeted risk-enrichment tools that maintain low alert rates among non-deteriorating patients.

To evaluate whether external knowledge retrieval could address this challenge, researchers Ching Yan Li, Tsz Kwan Wong, and Hang Sheung Wong conducted a retrospective, outcome-defined case-control investigation. The study was nested within all patient attendances triaged as Category 3 by nurses during 2025 at a single emergency department in Hong Kong. The investigators identified 118 cases who experienced death within 24 hours of emergency department registration or required direct admission to an adult or pediatric intensive care unit. For comparison, 472 endpoint-negative control attendances were selected through calendar-month-guided random sampling after applying prespecified exclusion criteria, establishing a fixed 1:4 case-to-control ratio before any model inference was performed.

The experimental setup evaluated two configurations of the DeepSeek-V3 architecture using identical information collected at the time of nursing triage. The baseline approach operated strictly as a prompt-only model, whereas the MECR-RAG framework incorporated retrieval augmentation. Specifically, MECR-RAG was designed to retrieve relevant sections from local clinical guidelines along with comparable historical patient cases recorded in 2024. Primary analytical metrics focused on linearly interpolated sensitivity at an operational threshold constrained to a 10% alert burden among endpoint-negative controls, alongside the partial area under the receiver operating characteristic curve calculated between false-positive rates of 0 and 0.10.

Across the cohort of 590 attendances, MECR-RAG achieved substantially higher discrimination at the constrained alert burden than its prompt-only counterpart. When evaluated at the 10% endpoint-negative alert threshold, MECR-RAG demonstrated a linearly interpolated sensitivity of 68.1% (95% confidence interval, 56.5% to 76.3%), whereas the baseline prompt-only model attained a sensitivity of only 27.8% (95% confidence interval, 23.9% to 32.5%). In parallel, the partial area under the curve across the 0 to 0.10 range was 0.0408 (95% confidence interval, 0.0321 to 0.0508) for MECR-RAG compared to 0.0152 (95% confidence interval, 0.0124 to 0.0186) for the baseline system.

The authors also examined a fixed-threshold operational alert defined by a model prediction of Category 2 or higher urgency. Under this operational rule, MECR-RAG flagged 80 of the 118 cases (67.8%) and 43 of the 472 controls (9.1%), resulting in a positive likelihood ratio of 7.44 (95% confidence interval, 5.45 to 10.16). In contrast, the prompt-only baseline alerted on 104 of the 118 cases (88.1%) but also flagged 160 of the 472 controls (33.9%), yielding a substantially lower positive likelihood ratio of 2.60 (95% confidence interval, 2.26 to 3.00), illustrating the baseline model’s propensity to generate high false-alert volumes.

In addition to evaluating the artificial intelligence models, the research team conducted a post hoc exploratory comparison against the National Early Warning Score 2 (NEWS2) when the score could be calculated from triage documentation. This complete-case analysis comprised 280 of the 590 attendances, representing 47.5% of the overall cohort. At the prespecified 10% endpoint-negative alert burden, NEWS2 achieved an interpolated sensitivity of 58.3%, compared to 54.2% for MECR-RAG, reflecting an absolute difference of −4.0 percentage points with a 95% confidence interval ranging from −21.8 to 13.7 percentage points.

The study authors emphasized that while retrieval augmentation enhanced discrimination over a basic large language model, it did not establish statistical or clinical superiority over standard NEWS2 scoring in the exploratory complete-case subset. Consequently, the incremental clinical value of the retrieval-augmented framework remains unproven. The researchers cautioned that their evaluation focused on a narrow composite outcome of rapid mortality and intensive care admission, meaning the findings do not demonstrate general triage accuracy, broader clinical utility, or readiness for direct deployment in emergency departments.

Subject of Research: Retrieval-augmented large language models for identifying severe early deterioration in Category 3 emergency department triage

Article Title: Low-burden identification of severe early deterioration among mid-acuity emergency department attendances initially triaged as category 3 using a retrieval-augmented large language model: a retrospective outcome-defined case-control study

Article References: Li, C. Y., Wong, T. K., & Wong, H. S. (2026). Low-burden identification of severe early deterioration among mid-acuity emergency department attendances initially triaged as category 3 using a retrieval-augmented large language model: a retrospective outcome-defined case-control study. BMC Emergency Medicine. https://doi.org/10.1186/s12873-026-01791-6

Image Credits: AI Generated

DOI: 10.1186/s12873-026-01791-6

Keywords: emergency department, retrieval-augmented generation, large language models, triage category 3, patient deterioration, case-control study, DeepSeek-V3, MECR-RAG, NEWS2, intensive care unit admission, emergency medicine, alert burden, partial AUC, positive likelihood ratio, mortality risk, clinical decision support, Hong Kong healthcare