A structured, platform-assisted workflow combining artificial intelligence drafting with mandatory faculty review significantly reduced the time required to finalize clinical examination questions while receiving higher expert ratings than traditional manual authoring, according to findings from a prospective paired comparison study conducted at a single dental college.
Single-best-answer questions represent a cornerstone of assessment across dental and medical education curricula, yet drafting high-quality items manually demands substantial time and effort from experienced faculty members. While automated and computational approaches have been suggested to ease this authoring burden, rigorous empirical evaluations examining structured generative tools deployed within operational assessment pipelines have remained scarce.
To investigate whether automated drafting could effectively support faculty in developing assessments from educational materials, researchers organized a prospective paired comparison trial registered with the Thai Clinical Trials Registry under identifier TCTR20260125010. Eight experienced dental faculty members authored assessment items across matched learning objectives using two distinct approaches: conventional manual drafting and a custom-built digital platform.
The experimental platform utilized the Google Gemini 2.5 Pro Application Programming Interface to generate initial question drafts structured within a standardized template. Rather than permitting autonomous item production, the system required faculty authors to mandatorily review, edit, and finalize each generated draft. To prevent order effects and cognitive priming from skewing comparative metrics, investigators counterbalanced the authoring sequence between the two workflows, producing 300 distinct clinical questions across 150 matched pairs.
Item-level finalization times, automatically tracked by the system, demonstrated substantial procedural efficiency gains. Manually authoring an assessment item required a mean of 6.0 minutes with a standard deviation of 1.2 minutes, whereas finalizing a platform-assisted item required a mean of 1.0 minute with a standard deviation of 1.4 minutes, reflecting a statistically significant reduction (p < 0.001). The recorded metric captured the complete drafting duration for manual questions and post-draft review and editing time for assisted items, excluding initial document upload, model generation latency, and discarded drafts.
Quality assessments conducted by three blinded subject-matter experts using a pilot-tested five-point Likert scale favored the platform-assisted pipeline. Finalized items generated with digital assistance achieved a mean expert quality rating of 4.7 with a standard deviation of 0.7, compared with a mean score of 4.1 with a standard deviation of 0.8 for items created entirely by hand, a difference that reached statistical significance (p < 0.001) under linear mixed-effects models that accounted for author clustering.
The study also analyzed descriptive item-level psychometric properties after deploying all 300 questions into formative assessments administered to undergraduate and postgraduate dental students. Platform-assisted questions demonstrated a mean difficulty index of 0.40, with 96 percent falling within the prespecified target range of 0.30 to 0.70, and a mean discrimination index of 0.24, with 94 percent meeting or exceeding the 0.20 threshold. In comparison, manually authored items showed a mean difficulty index of 0.46, with 80 percent within the target range, and a mean discrimination index of 0.26, with 81 percent meeting the discrimination threshold.
Despite these favorable findings, the authors emphasized key methodological boundaries, designating the research as an exploratory investigation at a single institution. Because the digital intervention unified automated drafting, structured templating, and expert human revision into a single pipeline, the trial did not isolate the model’s independent generative capability. Furthermore, because psychometric equivalence between authoring methods was not formally evaluated, the authors caution that the results must not be interpreted as evidence that artificial intelligence autonomously improves question quality or yields psychometrically equivalent examination items.
The prospective trial protocol was formally registered with the Thai Clinical Trials Registry under identifier TCTR20260125010 on January 25, 2026, defining the investigation’s parameters prior to participant recruitment.
Subject of Research: AI-assisted clinical question drafting in dental education
Article Title: Comparison of human versus AI-Assisted workflows for developing structured clinical single-best-answer questions in dental education
Article References: Islam, S., Gill, V., Qureshi, S., & Ali, M. (2026). Comparison of human versus AI-Assisted workflows for developing structured clinical single-best-answer questions in dental education. BMC Medical Education. https://doi.org/10.1186/s12909-026-10745-2
Image Credits: AI Generated
DOI: 10.1186/s12909-026-10745-2
Keywords: dental education, single best answer questions, assessment item development, generative AI, Google Gemini 2.5 Pro, faculty authoring workflows, formative assessments, item finalization time, expert quality ratings, item difficulty index, item discrimination index, prospective paired comparison, linear mixed-effects models, counterbalanced study design, standardized item templates, Thai Clinical Trials Registry
News Source: Courtney Benton. (November 14, 2026). AI-Assisted Workflow Cuts Dental Exam Question Drafting Time in Exploratory Trial. Scienmag.
