Binary Random Forest Models Match Complex Approaches for Predicting Hospital Bloodstream Infections

In a methodological investigation utilizing electronic health records, researchers determined that standard binary random forest models deliver predictive accuracy comparable to complex multinomial and competing risks frameworks when forecasting central line-associated bloodstream infections. Analyzing clinical data from 27,478 admissions and 30,862 central venous catheter episodes at University Hospitals Leuven, the research team examined whether incorporating time-to-event dynamics or competing endpoints improves prediction over a seven-day horizon. Hospitalized cohorts frequently experience competing events such as discharge or in-hospital mortality, yet analytical pipelines often overlook these occurrences or mismanage them through conventional survival methods.

The study evaluated four mathematical operationalizations of patient outcomes to predict central line-associated bloodstream infections occurring within seven days of assessment. These formulations comprised a binary classification model contrasting infection against no infection, a multinomial model differentiating between bloodstream infection, patient discharge, in-hospital death, or no qualifying event, a classical time-to-event survival model tracking duration until infection, and a competing risks framework modeling time to infection alongside discharge and death. Across 30,862 catheter episodes, investigators recorded 970 confirmed bloodstream infections, 1,466 patient deaths, and 28,426 hospital discharges, capturing clinical settings where benign removals and mortality far outnumber infectious complications. By testing these formulations against identical records, the authors aimed to determine how structural treatments of competing endpoints influence prediction performance, hyperparameter tuning efficiency, and individual risk estimation across inpatient populations.

To capture how clinical risk evolves over the duration of intravenous catheter placement, the investigators constructed both static and dynamic random forest models. Static models utilized clinical information available exclusively at the initial onset of the catheter episode, establishing a baseline assessment for infection risk during the initial seven days of line placement. In contrast, dynamic models updated patient predictions daily across a thirty-day observational window, systematically re-evaluating risk profiles at sequential daily landmark intervals from landmark zero through landmark thirty. All modeling configurations were trained and validated across one hundred independent random training and testing data splits to ensure robust statistical evaluation and minimize sampling bias.

Across primary discrimination metrics, binary classification, multinomial, and competing risks models demonstrated remarkably similar performance for static and dynamic forecasting intervals. For static predictions formulated at catheter placement onset, all three model families attained an identical area under the receiver operating characteristic curve of 0.74. Within the dynamic modeling framework, predictive discrimination steadily increased across early monitoring, reaching an optimal peak area under the curve of 0.77 at landmark day five before gradually declining across subsequent landmark intervals. Throughout all landmark assessments, binary models consistently matched the discriminative capacity of multi-state competing risks models, indicating that the computational burden of tracking competing event transitions yielded no measurable improvement in identifying patients who subsequently developed central line-associated bloodstream infections within seven days.

In contrast to binary and competing risks formulations, standard survival models that censored competing events at their exact time of occurrence exhibited severe predictive shortcomings. In static evaluations, survival models with event-time censoring produced pronounced risk overestimation, yielding median expected-to-observed ratios of 1.44 for seven-day administrative censoring and 1.47 for thirty-day administrative censoring, compared to balanced calibration ratios between 0.99 and 1.00 across alternative models. Furthermore, their discriminative performance lagged behind, posting median area under the curve scores of 0.729 and 0.724 respectively, whereas competing models achieved scores between 0.735 and 0.742. These distortions demonstrated that standard survival algorithms should be avoided when competing inpatient events occur.

The researchers discovered that this calibration failure could be resolved by modifying how survival models handle competing endpoints within the risk set. When individuals experiencing competing events were retained in the risk calculation until the specific prediction horizon rather than being censored immediately upon discharge or death, model performance became comparable to competing risks and binary structures. This horizon-censoring adjustment aligned static survival models with the broader cohort, restoring the median expected-to-observed calibration ratio to 1.00 and elevating discrimination metrics to 0.742 for seven-day models and 0.739 for thirty-day models. A corresponding pattern emerged across dynamic landmark evaluations, where unadjusted survival models suffered from elevated expected-to-observed ratios of 1.58 at landmark zero and 1.35 at landmark five, alongside depressed discrimination. Adjusting the censoring horizon eliminated this calibration gap, mirroring principles underlying Fine-Gray cumulative incidence function estimation.

Despite generating comparable overall discrimination metrics, underlying decision tree structures revealed profound differences in internal feature prioritization between model architectures. By analyzing the minimal depth of the maximal subtree, which identifies the tree depth at which a predictor variable is first split across the random forest, the authors observed distinct split selections driven by outcome operationalization. Models that evaluated all outcome levels simultaneously, such as multinomial and competing risks architectures, prioritized features associated with overall patient illness and disposition, selecting chemotherapy, systemic antibiotic administration, and C-reactive protein levels at earliest tree nodes. Conversely, binary classification and survival models focused strictly on infection endpoints consistently selected total parenteral nutrition at root and early splits. In dynamic models, multi-outcome architectures emphasized antibiotics, non-bloodstream infections, and intensive care unit admission, while binary models maintained early reliance on total parenteral nutrition.

These architectural differences also produced noticeable variations in individual risk predictions and precision-recall dynamics across the patient cohort. Multi-outcome models that incorporated discharge and mortality endpoints tended to predict higher absolute risk probabilities for high-risk patients, resulting in elevated area under the precision-recall curve values alongside slight miscalibration on the Estimated Calibration Index. Because central line-associated bloodstream infections represent low-prevalence outcomes, precision-recall metrics place substantial mathematical weight on discriminative capacity within the highest predicted risk percentiles. The investigators noted that this phenomenon reflects between-model variability, wherein distinct model specifications assign diverging risk scores to individual patients despite presenting identical global discrimination. Similar findings from prior cardiovascular literature demonstrated that misspecified models ignoring censoring produce conservative, depressed risk predictions, whereas correctly parameterized survival frameworks generate higher risk assignments without altering area under the curve metrics.

From a practical computational perspective, simpler modeling approaches provided substantial efficiency benefits over complex time-to-event architectures during training and hyperparameter optimization. Tuning runtimes scaled sharply with model complexity, with binary classification, multinomial frameworks, and horizon-censored survival models requiring the shortest tuning durations for static and dynamic implementations. Extending administrative censoring horizons to thirty days increased tuning times in static models, while competing risks models consistently demanded the longest optimization periods. Once hyperparameter tuning was finalized, however, execution times for final model assembly showed narrower variance across algorithms. Furthermore, operational inference speeds remained exceptionally rapid across all tested configurations, requiring a median runtime of under one second to generate predictions for an entire static test cohort of over 10,000 observations, and under two seconds for dynamic evaluation sets encompassing over 80,000 observations.

The study documented important software constraints encountered when implementing advanced random forest algorithms on large electronic health record datasets using the randomforestSRC package in R. The researchers highlighted that default tuning utilities in randomforestSRC restrict objective loss function customization and do not support tunable subsample sizes, prompting the team to employ the mlrMBO package for flexible Bayesian optimization. Furthermore, because the package lacks native capabilities for user-defined in-bag and out-of-bag samples of unequal sizes across clustered or dynamic data, the authors adjusted in-bag sizes to minimum thresholds to ensure computational tractability. Although this adjustment technically introduced a minor test leak, comparative validations against the ranger software library confirmed that its practical impact on predictive performance was negligible. To further overcome computational barriers in dynamic competing risks, the authors discretized event times and enforced administrative censoring horizons.

Comparing their findings against a companion investigation evaluating parametric regression models on the identical Leuven dataset, the authors observed consistent patterns alongside distinct algorithmic differences. While both studies concluded that binary, multinomial, and competing risks models achieve comparable performance, random forests delivered superior discrimination, reaching static area under the curve values of 0.742 and dynamic values of 0.775 at landmark five, compared to 0.721 and 0.747 for regression models. The authors emphasized several study limitations, noting that their work represents a methodological comparison rather than a validated clinical tool, and did not account for residual correlation among repeated catheter episodes within individual admissions. Furthermore, the analysis focused strictly on seven-day infection forecasts, acknowledging that survival and competing risks models retain clear practical utility when clinicians require simultaneous risk projections across multiple distinct time horizons without training separate binary algorithms.

Subject of Research: Random forest modeling approaches for central line-associated bloodstream infection prediction

Article Title: A comparison of modeling approaches for static and dynamic prediction of central line-associated bloodstream infections using electronic health records (part 2): random forest models

Article References: Albu, E., Gao, S., Stijnen, P., Rademakers, F. E., Janssens, C., Cossey, V., Debaveye, Y., Wynants, L., & Van Calster, B. (2025). A comparison of modeling approaches for static and dynamic prediction of central line-associated bloodstream infections using electronic health records (part 2): random forest models. Diagnostic and Prognostic Research, 9(1), Article 21. https://doi.org/10.1186/s41512-025-00194-8

Image Credits: AI Generated

DOI: 10.1186/s41512-025-00194-8

Keywords: central line-associated bloodstream infections, random forest models, electronic health records, predictive performance, competing risks, survival models, binary classification, multinomial models, dynamic prediction, AUROC, landmark modeling, University Hospitals Leuven, calibration, randomforestSRC, AUPRC, total parenteral nutrition, catheter episodes