A pilot evaluation of a newly developed intraoperative assessment tool designed for omentectomy training showed high internal consistency and inter-rater reliability during simulated surgical workshops, though it failed to differentiate trainee experience levels. The Omentectomy Assessment Tool Score, or OATS, was designed to evaluate obstetrics and gynecology trainees as they learn to execute an omentectomy. The procedure involves resecting the omentum, a protective peritoneal layer frequently implicated in gynecological malignancies, where complete surgical resection correlates directly with improved survival outcomes.
Cancer accounts for approximately thirty percent of global deaths from non-communicable diseases, a broader category of illness that continues to increase in burden worldwide. Because gynecological malignancies frequently spread to the peritoneal cavity and involve the omentum, oncologic surgical management often necessitates performing an omentectomy. Despite the clinical importance of complete surgical excision for patient survival, educators previously lacked a standardized instrument to evaluate whether gynecological surgical trainees have acquired the competence required to carry out this essential operative procedure.
To address this educational gap, investigators based in the Department of Obstetrics and Gynecology at Aga Khan University developed the OATS tool and conducted a pilot validation study. Educational sessions featuring simulation-based, hands-on workshops were hosted across two institutional clinical environments, specifically the Aga Khan University Hospital and the Jinnah Postgraduate Medical Centre. The study enrolled forty-three obstetrics and gynecology trainees to perform the surgical task within these specialized simulation workshops under direct observational evaluation.
The evaluation process relied on Messick’s unitary validity framework to structure the collection of validity evidence for the newly designed instrument. During the simulation workshops, two clinical raters simultaneously and independently scored each of the forty-three participating trainees across an eighteen-item assessment rubric. Each item on the OATS instrument was graded using a seven-point adjectival scale ranging from zero to six, allowing observers to evaluate multiple specific dimensions of operative technique throughout the simulated omentectomy.
Statistical evaluation showed promising metrics for reliability and internal structure across the simulated exercises. The internal consistency of the complete eighteen-item tool reached an overall Cronbach’s alpha of 0.94. Inter-rater reliability, determined by intraclass correlation coefficients, demonstrated substantial rater agreement. Thirteen of the eighteen assessment items achieved intraclass correlation coefficients ranging between 0.75 and 0.84, while the remaining five items yielded values between 0.64 and 0.74, with no item falling below an intraclass correlation of 0.5.
Despite these positive reliability measures, the assessment tool was unable to demonstrate discriminant validity across different stages of medical training. Researchers initially planned to evaluate discriminant power across distinct residency training years using the Kruskal-Wallis test. However, due to small trainee counts within individual training years, the investigators collapsed all residents into a single combined cohort and compared their scores against advanced surgical fellows using the non-parametric Mann-Whitney U test, adopting a significance threshold of p < 0.05.
The comparative analysis revealed no statistically significant difference between the mean scores achieved by the different trainee cohorts. Both resident trainees and fellows scored near the mid-point of the seven-point adjectival scale. The study authors noted that this absence of discriminant validity might stem from the small pilot sample size or potential constraints in the physical fidelity of the simulation model. Consequently, researchers concluded that utilizing OATS for credentialing is premature until larger multicenter investigations validate the instrument within live operative environments.
Subject of Research: Surgical performance assessment tool for simulated omentectomy training
Article Title: Development and validation of the Omentectomy Assessment Tool Score (OATS) – a novel intraoperative performance assessment tool for surgical education
Article References: Chishti, U., Afzal, A. S., Akhtar, M., Aziz, A. B., Raza, A., & Babar, S. (2026). Development and validation of the Omentectomy Assessment Tool Score (OATS) – a novel intraoperative performance assessment tool for surgical education. BMC Medical Education. https://doi.org/10.1186/s12909-026-10649-1
Image Credits: AI Generated
DOI: 10.1186/s12909-026-10649-1
Keywords: omentectomy, surgical education, performance assessment, Omentectomy Assessment Tool Score, Messick validity framework, simulation training, inter-rater reliability, internal consistency, gynecologic oncology, obstetrics and gynecology, Aga Khan University, Jinnah Postgraduate Medical Centre, intraclass correlation, Cronbachs alpha, discriminant validity, surgical simulation, clinical competence, trainee assessment
News Source: Courtney Benton. (November 16, 2026). Simulation Tool for Omentectomy Demonstrates Reliability but Lacks Trainee Discrimination. Scienmag.



