Machine learning with patient words sharpens emergency triage classification

A retrospective cohort study from Brazil reports that machine learning models classify emergency department triage more accurately when they are fed not only structured clinical data but also the free-text words patients use to describe their complaints. The research, published in BMC Health Services Research, combined machine learning classifiers with natural language processing to improve triage classification under the Emergency Severity Index, evaluated across three triage levels.

The study addresses known weaknesses of traditional triage systems, which have been associated with under-triage and over-triage as well as high variability and individual bias. Precise stratification at this initial assessment stage matters because it can reduce patient safety risks, determine optimal healthcare treatments, and optimize costs. The team, led by Bruno Matos Porto and colleagues, built a classification framework testing three data configurations: structured data only, unstructured data only, and a combination of both, using records from the emergency department of a large private hospital in Brazil.

The technical core of the work lies in how free-text chief complaints were converted into numerical features. The researchers used term frequency–inverse document frequency, or TF-IDF, a vectorization method that weighs words by how often they appear in a given complaint and how distinctive they are across the corpus. Four classification algorithms were compared: eXtreme Gradient Boosting (XGBoost), Random Forest, a Multilayer Perceptron neural network, and Logistic Regression. Because triage data are imbalanced across acuity levels, the team also tested two oversampling techniques, SMOTE and ADASYN, which synthesize new examples of minority classes.

Model development followed a defined pipeline. A feature selection step based on variable importance plots was performed for all analysis sets, hyperparameters were optimized by grid search on the training set, and performance was evaluated on the validation set. A 10-fold cross-validation was run on the training and validation sets, and eight performance metrics were calculated. The dataset covered 64,382 adult patients, of whom 38.3 percent were triaged at levels 1 and 2.

The highest ROC-AUC came from XGBoost using structured plus unstructured data, which achieved 0.871 on the validation set and 0.873 on the test set. Algorithms using the complete datasets generally outperformed those using only structured data, and overall, models that incorporated natural language processing achieved the highest ROC-AUC. The NLP model used to vectorize patients’ chief complaints enhanced performance metrics by 4 to 11 percent on both the validation and test sets.

The authors conclude that integrating patients’ chief complaints through NLP enhanced the performance of machine learning algorithms for patient triage in the ESI system in Brazil, and that the inclusion of unstructured data resulted in more accurate and reliable triage predictions. In practical terms, the framework treats the phrasing of a chief complaint as a model input alongside structured variables, recovering information that structured fields alone do not capture.

Caution is warranted before generalizing these results. The study is retrospective and drawn from a single large private hospital in Brazil, so the models were trained and validated on one institution’s population and documentation practices. The framework classifies across three levels of the ESI system rather than the full granularity of the scale, and the reported results come from validation and test sets rather than from a live deployment in which algorithms and triage staff work side by side.

Within those limits, the study offers a concrete recipe for augmenting triage classification: combine structured data with the patient’s own words, handle class imbalance deliberately with oversampling techniques, and let gradient-boosted classifiers do the heavy lifting. The result is a model that, trained on more than sixty thousand prior visits, distinguishes acuity levels with roughly 87 percent ranking probability on held-out test data.

Subject of Research: Machine learning and natural language processing for emergency department triage classification

Article Title: Improving patient triage classification in the emergency department using natural language processing and machine learning algorithms: a retrospective cohort study

Article References: Porto, B. M., Fogliatto, F. S., Wolf, J. M., Pille, A., & Schmitz, P. S. K. (2026). Improving patient triage classification in the emergency department using natural language processing and machine learning algorithms: a retrospective cohort study. BMC Health Services Research. https://doi.org/10.1186/s12913-026-15912-1

Image Credits: AI Generated

DOI: 10.1186/s12913-026-15912-1

Keywords: emergency medicine, triage, machine learning, natural language processing, XGBoost, TF-IDF, Emergency Severity Index, SMOTE, ADASYN, Brazil, retrospective cohort study, patient safety