A systematic review published in Diagnostic and Prognostic Research indicates that machine learning-based prognostic models for COVID-19 exhibit significantly lower reporting quality than conventional statistical models. The study, led by Ioannis Partheniadis and colleagues, evaluated 53 studies that described 71 distinct prognostic models. By applying the TRIPOD checklist to conventional models and the newer TRIPOD + AI extension to machine learning models, the researchers found that adherence to reporting standards was generally poor across both groups. However, the gap between the two methodologies was substantial, with machine learning studies showing a mean adherence rate of 28.4% compared to 38.1% for conventional models.
The primary objective of the research was to assess the completeness of reporting in prognostic prediction models for COVID-19. Prognostic models are statistical tools used to estimate the risk of specific outcomes, such as severe illness or mortality, based on patient characteristics. During the pandemic, there was an unprecedented surge in the development of such models, driven by the availability of large datasets and accessible software. While these models hold potential for guiding clinical decisions and optimizing resource distribution, their utility is heavily dependent on transparent and rigorous reporting. The authors noted that incomplete reporting hampers the ability of other researchers to assess how a study was conducted, interpret its findings, or independently validate the models for clinical use.
To conduct this evaluation, the team performed literature searches in MEDLINE, Epistemonikos.org, and Scopus up to July 31, 2024. They identified 1,529 records from 17 systematic reviews, which were then screened to identify primary studies that reported model development, internal validation, and external validation. After removing duplicates and conducting full-text reviews, 53 studies met the eligibility criteria. The protocol for this systematic review was prospectively registered on the Open Science Framework, ensuring that the methodology was predefined before the data extraction began. This rigorous selection process allowed the researchers to focus on models that had undergone the full cycle of development and validation, providing a clearer picture of reporting practices in more mature studies.
The 71 included models were nearly evenly split, with 58.5% being traditional statistical models and 41.5% being machine learning-based. The models predominantly focused on hospitalized patients with confirmed COVID-19, with most data coming from single-country sources, particularly China and the United States. Common predictors included age, C-reactive protein, lactic dehydrogenase, lymphocyte count, and oxygen saturation. The most frequently predicted outcomes were in-hospital mortality and severe or critical illness. Most studies were retrospective in design, and a significant proportion used multicenter data. The median participant age across the models was 61 years, with an average proportion of male participants at 54.8%.
When assessing adherence to the TRIPOD and TRIPOD + AI guidelines, the researchers found that no study fully adhered to the abstract reporting requirements. Appropriate titles were included in only a minority of cases, with 29.0% of conventional model studies and 13.6% of machine learning studies meeting the criteria. Sample size calculations were not fully reported in any of the included studies, a critical omission that obscures the risk of overfitting. Reporting of methods and results sections was poor across both frameworks, with significant deficiencies in areas such as data preprocessing, clear definitions of predictors and outcomes, and strategies for handling missing data. These omissions are particularly concerning because they limit the reproducibility of the models and make it difficult for clinicians to determine whether a model is suitable for their specific patient population.
Significant differences in adherence were observed in several key methodological domains. For instance, reporting on the source of data was substantially better in conventional model studies, with 90.3% adherence compared to only 22.7% in machine learning studies. Conversely, predictor selection and pre-processing were less thoroughly reported in conventional studies (6.5%) than in machine learning studies (50.0%). Adherence to items detailing predictor definitions, measurements, and blinding was extremely poor across both frameworks, with no meaningful difference between the two.
Not a single study provided a justification or calculation for sample size, a fundamental element of study design. Model performance stability reporting, which is required by the TRIPOD + AI guidelines, had 0.0% adherence, with machine learning studies failing to report on performance variability or bootstrap-based model evaluations. Protocol registration and access also showed 0.0% adherence in the machine learning group.
The overall mean adherence across all studies was 34.07%, with conventional statistical models achieving a higher mean adherence of 38.11% compared to 28.37% for machine learning approaches. The difference was statistically significant, with a p-value of 0.0012. The authors noted that this discrepancy is unsurprising given that the TRIPOD + AI guidelines were only released in April 2024, two years after the latest included study in the review. Many of the machine learning studies were published before the specific AI-focused reporting standards were available, which may have contributed to the lower adherence scores.
Less than half of the studies made their final model fully available, with only 12.9% meeting the minimum requirements of the TRIPOD statement and a mere 4.6% satisfying the TRIPOD + AI requirements. This lack of transparency poses serious barriers to replication and external validation, which are essential for confirming the robustness of a prediction model. The authors point out that inadequate detail on missing-data methods, lack of optimism correction in performance estimates, and insufficient calibration and validation approaches further undermine confidence in the reported performance metrics. Many studies reported C-index values that appeared unrealistically high, a phenomenon often associated with overfitting and inadequate validation. Without access to the full model and detailed reporting, it is impossible for other researchers to verify these claims or adapt the models for different clinical contexts.
The study has several limitations that should be considered when interpreting the findings. Reporting quality was assessed by a single reviewer, although there was good agreement in a subsample that was independently assessed by a second reviewer. The evaluation focused only on the best-performing model from each study, which may underestimate broader reporting deficiencies if other models within the same study were poorly reported. Additionally, different reporting frameworks were applied to conventional and machine learning models, which may influence overall adherence estimates due to structural differences between the checklists. The authors acknowledge that reporting guidelines are not formal tools for evaluating study quality, but rather standards for transparency. Therefore, low adherence does not necessarily indicate that the models are invalid, but it does suggest that the evidence supporting them is less accessible and verifiable.
These findings align with previous reviews showing poor adherence to TRIPOD guidelines in COVID-19 and other biomedical fields. Earlier meta-research has highlighted widespread reporting deficiencies and high bias risk across COVID-19 prediction studies. The lower adherence observed in this study compared to some previous reviews may be partly explained by the stricter inclusion criteria, which focused only on studies that reported development, internal validation, and external validation. The application of the TRIPOD + AI update, which introduces more specific reporting expectations for AI-based models, may also have contributed to lower adherence values.
To improve the reliability and usability of prediction models, researchers must rigorously apply TRIPOD or TRIPOD + AI in their reporting. Journal editors and peer reviewers should insist on these guidelines, as only a small percentage of journals in the sample explicitly recommended their use. The authors suggest that raising awareness among researchers about the importance of complete and transparent reporting is essential. Continued meta-research and stronger editorial policies are needed to enforce compliance. Additionally, reporting guidelines themselves should be refined by moving beyond expert opinion to incorporate structured methodological frameworks. Future studies should consistently follow the latest guidelines, and journals and peer reviewers should take a more active role in enforcing their use to ensure that prediction models are developed and reported with the rigor required for clinical application.
Subject of Research: Reporting quality of machine learning and conventional prognostic models for COVID-19
Article Title: Machine learning-based COVID-19 prognostic models lag behind in reporting quality: findings from a TRIPOD/TRIPOD + AI systematic review
Article References: Partheniadis, I., Talimtzi, P., Nikolakopoulou, A., & Haidich, A.-B. (2026). Machine learning-based COVID-19 prognostic models lag behind in reporting quality: findings from a TRIPOD/TRIPOD + AI systematic review. Diagnostic and Prognostic Research, 10(1), Article 3. https://doi.org/10.1186/s41512-026-00218-x
Image Credits: AI Generated
DOI: 10.1186/s41512-026-00218-x
Keywords: Machine Learning, COVID-19, Prognostic Models, TRIPOD, Systematic Review, Reporting Quality, Clinical Prediction, Transparency, Medical Research, Artificial Intelligence, Machine, learning-based
News Source: Blake Davidson. (November 8, 2026). Machine Learning Models Show Lower Reporting Quality in COVID-19 Research. Scienmag.
