An artificial intelligence model developed to evaluate children performing the forward roll demonstrated substantial to almost perfect agreement with expert consensus ratings and significantly reduced human scoring error when utilized as an assessment aid, according to research published in BMC Sports Science, Medicine and Rehabilitation. Investigating the Fundamental Motor Skills in Sport (FUS) assessment system, the international study examined whether automating criterion-level evaluations could support movement instruction and long-term population-level monitoring, particularly given that low proficiency in fundamental motor skills among children highlights an urgent requirement for valid, feasible, and dependable evaluation techniques.
To explore how machine learning could alleviate these practical constraints, the research team designed an investigation to determine whether artificial intelligence predictions could replicate expert consensus decisions on forward roll performances and whether delivering criterion-level algorithmic feedback could systematically elevate human scoring fidelity, boost inter-rater reliability, and reduce perceived evaluation workload.
To establish the baseline analytical accuracy of the automated system under standardized video conditions, the investigators compared the scores produced by the artificial intelligence model against established expert consensus ratings across a dataset of 66 forward roll video recordings. These sampled recordings were deliberately chosen to span the full spectrum of performance quality, covering the entire range of total scores from 0 to 5. When examining agreement across individual movement criteria, Cohen’s kappa values between the model and expert consensus ranged between 0.73 and 0.85, corresponding to substantial to almost perfect alignment. Furthermore, when evaluating aggregate performance across the full scoring scale, the quadratic weighted kappa reached 0.95, with a 95 percent confidence interval spanning from 0.92 to 0.97.
Building upon the standalone model validation, the researchers implemented a counterbalanced experimental trial to measure how artificial intelligence assistance influenced human scoring accuracy and cognitive demand during actual evaluations. A cohort of 20 raters recruited from Malta and Poland assessed two distinct, independent sets consisting of 22 forward roll videos each. Each rater performed one assessment session unaided and one session with real-time criterion-level artificial intelligence support. Human ratings generated across both experimental conditions were subsequently benchmarked against a separate, blinded expert consensus panel to measure absolute error and agreement, while inter-rater reliability was tracked descriptively across both national cohorts.
The introduction of criterion-level recommendations yielded clear improvements in scoring consistency, bringing human evaluations into significantly closer alignment with the separate blinded expert benchmark. Human evaluators working with artificial intelligence guidance achieved a mean absolute error of 0.68, with a 95 percent confidence interval of 0.53 to 0.85, in contrast to a mean absolute error of 0.99, with a 95 percent confidence interval of 0.85 to 1.13, observed during unassisted assessments. In addition, the quadratic weighted kappa measuring concordance with the blinded consensus rose from 0.60 (95 percent confidence interval 0.44 to 0.70) without support to 0.79 (95 percent confidence interval 0.65 to 0.87) with support. The reduction in scoring error proved remarkably consistent, benefiting 19 out of the 20 participating raters, a statistically significant shift confirmed by a Wilcoxon test yielding a p-value of less than 0.001, conditional on the sampled video recordings.
In addition to accuracy improvements, the experimental protocol evaluated consistency across raters and subjective evaluation strain. Descriptive point estimates for the two-way random-effects single-rater intraclass correlation coefficient, ICC(2,1), were consistently higher under the artificial intelligence condition across all four country-by-video-set experimental comparisons. To track subjective workload, raters completed the standardized NASA Task Load Index after finishing each condition. Following statistical adjustment via the Holm correction method to account for multiple comparisons across workload dimensions, perceived effort was the sole metric to demonstrate a statistically significant reduction under artificial intelligence support, recording a p-value of 0.045.
Despite these encouraging findings under standardized video recording conditions, the study authors outlined several essential caveats regarding the scope and generalizability of their conclusions. The participating evaluators in the experimental trial were master’s degree students, meaning the observed rater outcomes specifically describe this demographic rather than seasoned education professionals. Consequently, the investigators emphasized that extending criterion-level artificial intelligence recommendations as a decision aid for human-supervised scoring will require separate validation among practicing physical education teachers, across broader pediatric age groups, within diverse motor skills beyond the forward roll, and across routine, unconstrained school environments.
Subject of Research: Artificial intelligence assistance for forward roll assessment in fundamental motor skills testing
Article Title: Artificial intelligence-assisted assessment of the forward roll in the Fundamental Motor Skills in Sport (FUS) test: agreement with expert consensus, inter-rater reliability, and rater workload
Article References: Makaruk, H., Decelis, A., Makaruk, B., Tomaszewski, P., & Sadowski, J. (2026). Artificial intelligence-assisted assessment of the forward roll in the Fundamental Motor Skills in Sport (FUS) test: agreement with expert consensus, inter-rater reliability, and rater workload. BMC Sports Science, Medicine and Rehabilitation. https://doi.org/10.1186/s13102-026-02232-4
Image Credits: AI Generated
DOI: 10.1186/s13102-026-02232-4
Keywords: Fundamental Motor Skills in Sport, forward roll assessment, artificial intelligence, expert consensus, inter-rater reliability, rater workload, NASA Task Load Index, Cohen's kappa, quadratic weighted kappa, intraclass correlation coefficient, video analysis, motor skill proficiency, perceived effort, master's students, FUS test, criterion-level scoring, counterbalanced experiment, blinded evaluation
News Source: Ophelia Keating. (November 14, 2026). AI Support Improves Human Forward Roll Scoring Accuracy in Motor Skills Test. Scienmag.



