Graph-based AI tracks hospital quality shifts across CMS releases

A machine-learning framework built for publicly owned US hospitals has shown that it can hold onto its predictive accuracy as new data releases arrive, according to a study published in Frontiers in Public Health. The system, called Health–Insurance–Operational–Proxy synergy-aware Ordinal Graph Learning, or HIOP-GL, was tested on two distinct hospital performance outcomes drawn from the Centers for Medicare & Medicaid Services. Its clearest advantage was in maintaining ordinal agreement—getting the rank order of hospital performance tiers right—when evaluated on data releases that came after the model’s development was complete.

Development used 1,849 hospital-release observations from 341 hospitals, followed by 1,236 post-development observations from 338 hospitals across four later release pairs.

The architecture adapts to each outcome.

Across five independently fitted models, PH-Star achieved a quadratic weighted kappa of 0.816 plus or minus 0.009 on the prespecified temporal test, with Macro-F1 of 0.584 and tier-distance mean absolute error of 0.455. The five-seed ensemble reached QWK 0.824. On the four later post-development release pairs, QWK was 0.757 plus or minus 0.013 across fits and 0.768 for the ensemble, compared with 0.705 for CatBoost and 0.680 for random forest. Paired ensemble QWK differences were 0.057 versus CatBoost and 0.072 versus random forest, both with Holm-adjusted p-values of 0.002. Macro-F1 differences crossed zero, and the ensemble correctly classified 640 of 1,236 post-development observations, compared with 657 for CatBoost and 646 for random forest.

The FY2026 PH-VBP evaluation told a more complicated story. The two-year-refit model reached QWK 0.678 plus or minus 0.021 across five fits, with the ensemble at 0.717. HistGradientBoosting and CatBoost reached ensemble QWK values of 0.630 and 0.639, and the paired differences of 0.087 and 0.077 were both significant after Holm adjustment. Yet the evaluation occurred under substantial drift: mean TPS rose from 22.51 in FY2025 to 29.84 in FY2026, pushing 48.1 percent of FY2026 hospitals into tier 5 under the cutoffs fixed from FY2024. From August 2025 onward, only one of nine development insurance/cost variables remained observed in the CMS payment-and-value file, leaving an 88.9 percent cell-missing rate in that view. Performance below the tuning period must therefore be read in the context of both target and schema change.

The hospital-disjoint temporal ensemble, which predicted each FY2026 hospital only with models fitted without that facility’s historical records, reached QWK 0.689. Leave-Census-region-out prediction reached QWK 0.636, versus 0.524 and 0.562 for the continuous baselines, with paired differences of 0.111 and 0.072 both supported. These designs reduce direct hospital and regional overlap but remain internal to the same reporting system rather than validation by an independent data owner.

Among 229 California non-public acute-care hospitals with zero facility-ID overlap with the development panel, the ensemble achieved QWK 0.637, Macro-F1 0.441, and tier-distance MAE 0.721. This cohort changes ownership and facility identity while retaining the CMS feature and outcome system, so it estimates same-outcome ownership transport only.

Linking through the official California crosswalk, the expected PH-Star tier showed positive but modest concordance with the 2023 HCAI multi-measure performance criterion across 243 hospitals, with Spearman’s rho of 0.247 and a three-criterion Holm p-value of 0.0006. A 2024 HCAI mortality-specific sensitivity in 233 hospitals was weaker and not significant at rho 0.096. In New York, 57 exact-linked non-public hospitals showed correlation of 0.451 between expected tier and lower 2024 NYSDOH all-payer risk-adjusted mortality composites, with a three-criterion Holm p-value of 0.00096. Component results were heterogeneous: concordance was supported for mortality in selected conditions at rho 0.441 but not for selected procedures at rho 0.179. Across a conservative six-association multiplicity sensitivity, Holm-adjusted p-values were 0.0012 for California broad performance and 0.0024 for the New York exact-link, expanded-link, and IQI91 associations, while both unsupported mortality associations yielded 0.287. The exact-link cohort overrepresented critical-access hospitals and contained no psychiatric or children’s hospitals, and matched and unmatched eligible hospitals had similar model-score distributions, though hospital-type composition differed.

The study’s authors report that HIOP-GL’s advantage was most consistent for ordinal agreement under temporal transport, while several other metrics told a different story. Random forest had higher Macro-F1 of 0.682 versus 0.584 and lower tier-distance error of 0.324 versus 0.455 on the original temporal test, and TabPFN v1 had higher low-tier recall of 0.639 versus 0.600. In the fully hospital-grouped PH-Star evaluation, where every evaluated hospital was absent from its fold’s development data, pooled QWK was 0.661 for HIOP-GL versus 0.730 for ordinal logistic regression. In FY2026 PH-VBP, Macro-F1 was 0.402 for HIOP-GL versus 0.409 for HistGradientBoosting, and low-tier recall was 0.469 versus 0.568, with paired intervals crossing zero. The authors conclude that the supported advantage concerns QWK-focused retrospective stratification, that model choice should follow the decision metric and target cohort, and that prospective validation of the same outcomes across independent health systems remains the necessary next step toward operational use.

Subject of Research: Ordinal graph learning for stratifying publicly owned US hospitals on CMS quality and value-based purchasing outcomes

Article Title: Task-adaptive multi-view learning with ordinal graph and continuous-score paths for public hospital performance stratification

Article References: Gan, Y., & Li, D. (2026). Task-adaptive multi-view learning with ordinal graph and continuous-score paths for public hospital performance stratification. Frontiers in Public Health, 14, Article 1890759. https://doi.org/10.3389/fpubh.2026.1890759

Image Credits: AI Generated

DOI: 10.3389/fpubh.2026.1890759

Keywords: hospital quality, CMS ratings, ordinal classification, graph learning, value-based purchasing, machine learning, predictive modeling, public hospitals, temporal validation, quadratic weighted kappa, health services research, performance stratification