AI lectures fall short of human teaching in urology residency trial

A nationwide randomized controlled trial has found that lectures generated entirely by artificial intelligence did not improve exam performance among urology residents, even though many of the trainees remain enthusiastic about blending AI into their future education.

The headline result was a null finding: post-education exam scores were not significantly different across the three groups. Residents who watched only AI-generated lectures, those who watched only traditional lectures, and those who watched both performed similarly on the exam. This suggests that, at least for short-term knowledge acquisition measured by this test, replacing a human lecturer with an AI-generated lecture did not measurably change outcomes.

However, the residents’ subjective experience told a different story. Traditional lectures were rated significantly better than AI-generated lectures in terms of informativeness, content quality, narration, and the ability to sustain attention, with p-values below 0.01. Although residents reported high expectations of the technology, only 27 percent actually found the AI-generated education beneficial. The study’s conclusion states that AI-generated lectures produced by non-specialized, popular large language models were regarded as inferior by residents in terms of content quality, reliability, and educational value.

The exam questions themselves also revealed a difference between human and AI output. Supervisor-selected AI-generated exam questions were significantly easier than traditionally generated questions and were more frequently classified as “too easy” (p = 0.01). The study’s keypoint section notes that AI-generated items had a higher adjusted probability of a correct response than traditionally generated items, indicating that the AI-written questions did not challenge the residents to the same degree as those written by human educators.

Despite these critical assessments, the residents were not ready to abandon AI altogether. After completing the education, 64 percent of residents favoured integrating AI into a hybrid educational model. This gap between the perceived quality of current AI lectures and the appetite for AI-supported learning formats is one of the most striking aspects of the findings: trainees judged the existing technology harshly while still expressing optimism about its future role.

The authors conclude that general-purpose large language models are not ready to deliver residency-level education on their own, although hybrid AI-supported models may represent a promising future direction. They suggest that the development of domain-specific, urologically trained large language models could provide the higher quality AI-generated content that residency-level medical education requires. In other words, the problem may lie less with the concept of AI teaching than with the generic, non-specialized models currently available.

The study is an early-access, peer-reviewed article published on 27 October 2026. Within those limits, the trial offers a concrete data point in an ongoing debate: AI can match human lectures on test scores, but residents still judge the human classroom as the better learning experience, and the exam questions AI writes today remain too easy to trust for high-stakes assessment.

Subject of Research: A randomized controlled trial comparing AI-generated, traditional, and hybrid lectures in urology residency education

Article Title: Is AI ready for theoretical residency education? Challenging AI in a comparative effectiveness research study

Article References: Göger, Y. E., Tarhan, H., Akpinar, C., Sarikaya, E. A., & Esen, A. A. (2026). Is AI ready for theoretical residency education? Challenging AI in a comparative effectiveness research study. BMC Medical Education. https://doi.org/10.1186/s12909-026-10583-2

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10583-2

Keywords: artificial intelligence, medical education, urology residency, large language models, randomized controlled trial, hybrid education, AI-generated lectures, exam performance, residency training, BMC Medical Education, ready, theoretical