Generative artificial intelligence has moved from novelty to infrastructure in education research, according to a scoping review published in the journal AI, Brain and Child. The study, led by Jiutong Luo, Zijing Tang and Meng Sun, analyzed empirical research presented at four premier technology-oriented educational conferences: the International Conference on Artificial Intelligence in Education (AIED), the ACM Learning at Scale Conference (L@S), the International Learning Analytics and Knowledge Conference (LAK) and the International Conference on Educational Data Mining (EDM). Because computer science communities, including AI research, traditionally report new system designs and early empirical validations at conferences rather than in journals, the authors argue that conference proceedings offer the timeliest window into how generative AI is actually being built, tested and deployed in classrooms and learning platforms.
The team screened 1,032 conference papers published between 2023 and 2025, retrieved manually from the official proceedings tables of contents as of November 15, 2025. After keyword filtering for terms such as ChatGPT, large language model and generative AI, followed by independent manual screening, 189 empirical studies were retained for in-depth analysis. Poster papers, demo abstracts and workshop proposals were excluded because they lack the methodological detail needed for systematic coding, and studies that only surveyed perceptions of generative AI without actual interaction were also left out. Two researchers coded the dataset independently, achieving a Cohen’s Kappa of 0.88, which indicates strong agreement, and disagreements were resolved through discussion until consensus.
The growth curve is striking. Publication volume expanded from just five generative AI studies in 2023 to 133 in 2025, with the AIED conference alone hosting 90 studies in 2025. The authors describe this as the end of the novelty phase, where the mere presence of generative AI was noteworthy, and the beginning of an integration phase focused on sustainable, pedagogically grounded implementation. Early papers in 2023 and 2024 tended toward broad discussions of ChatGPT, opportunities and challenges, while 2025 publications are far more granular, targeting specific functions such as feedback generation, distractor generation, simulated students and retrieval-augmented generation.
Geographically, the research is globally distributed but heavily concentrated. The United States dominates in volume, producing work ranging from automated evaluation of peer feedback quality in higher education to fine-grained analysis of math tutoring dialogues. China, including Hong Kong SAR, shows a strong emphasis on scalable assessment and support systems, particularly automated scoring for English as a Foreign Language education and academic writing support. Germany contributes studies centered on teacher education and language acquisition, including AI tools to foster self-reflection in teacher seminars. Other contributions come from Australia, Japan, Israel, Brazil, Spain, the Netherlands, Switzerland, the United Kingdom, Luxembourg, Morocco, India and Singapore, with regional specializations such as Brazil’s exploration of offline language models for resource-constrained environments. The authors caution that this concentration raises concerns about equity and the generalizability of generative AI applications across diverse educational environments.
A defining characteristic of the field is the dual status of generative AI as both a tool for learning and a subject of investigation. In a substantial number of studies, the models themselves were treated as participants, benchmarked for performance, bias and reliability. This GenAI-as-participant paradigm allows researchers to simulate educational scenarios at scale without the ethical and logistical constraints of human-subject research. Examples include instantiating generated students to test the validity of assessment items, benchmarking multiple large language models on their ability to generate effective distractors for programming challenges, and evaluating automated grading, detecting AI-generated content and simulating patient interactions for pharmacy training.
When human learners were involved, higher education remained the primary testbed, providing a convenient and data-rich context for complex tasks like programming and academic writing. Applications in K-12 settings are maturing but remain comparatively underrepresented, a gap the authors flag as a developmental blind spot. Within the K-12 subset, studies concentrated on assessment functions such as automated short-answer grading, rubric-based essay scoring and formative feedback, alongside teacher-facing monitoring and classroom analytics tools and tutoring systems in mathematics and programming. The authors emphasize that children and adolescents are still developing metacognitive regulation, epistemic judgment and social-emotional awareness, so interactions with highly fluent and socially responsive AI systems may increase the likelihood of over-reliance, attribution of authority or reduced critical evaluation of outputs. Recent work on caregiver-supported homework environments suggests adult mediation plays a crucial role in shaping how younger learners interpret AI-generated feedback.
On the technical side, the landscape is dominated by proprietary models from the GPT family, with ChatGPT versions 3.5 and 4.0 serving as the primary engine in the vast majority of studies. However, a diversification trend is evident: researchers increasingly adopt open-source models like LLaMA and Mistral for privacy and control, proprietary alternatives such as Claude and Gemini for reasoning and multimodal capabilities, regional models like Qwen for Chinese language contexts, and offline-capable models like Gemma for resource-constrained settings. Prompt engineering remains the ubiquitous baseline approach due to its low barrier to entry, but the review documents growing integration of retrieval-augmented generation, which grounds outputs in external knowledge bases such as course materials; fine-tuning, applied in tasks requiring adherence to specific rubrics or output formats; reinforcement learning to align model outputs with educational goals; and multi-agent systems in which multiple language model instances collaborate, decomposing tasks between student agents and instructor agents.
In terms of application domains, computer science and programming is the most prolific field with 54 papers, spanning introductory Python tutoring, collaborative pair programming and automated syntax feedback. Mathematics follows with 49 papers focused on step-by-step tutoring and reliable grading. Language learning accounts for 24 papers, with further contributions in physics and natural sciences, business and social sciences, and medicine. The authors identify a thematic shift from content generation toward process-oriented learning support: rather than just correcting code, new systems prompt students to reflect on their errors or engage in Socratic dialogue, and learning-by-teaching models let students teach a simulated AI peer. Automated grading and feedback generation nonetheless continue to dominate many contexts, particularly in large-scale or resource-constrained environments.
Methodologically, the reviewed studies show a marked turn toward empirical rigor. Experimental designs now dominate, with randomized controlled trials increasingly common, and studies benchmark generative AI systems against human instructors, alternative baselines or predefined pedagogical objectives. Mixed-method approaches triangulate quantitative metrics with qualitative insights, and advanced psychometric techniques, including Quadratic Weighted Kappa for assessing agreement between human and AI graders, are becoming standard. The central research question, the authors note, has evolved from whether generative AI can perform a task to how effectively it performs under controlled conditions.
The review also identifies persistent limitations in the evidence base itself. There is a critical longitudinal gap: most studies measure immediate performance metrics such as grading accuracy, while very little is known about the long-term effects of AI mentorship on student agency, motivation and skill retention, and the authors warn of a genuine risk of cognitive offloading, where students become dependent on AI for critical thinking tasks. They call for longitudinal designs that track learners over semesters rather than sessions, greater representation of K-12 and Global South contexts, and rigorous auditing of agentic systems for behavioral alignment and bias. The review’s own scope is limited to four conferences and to full and short papers, so it may underrepresent more mature journal-based studies and contributions in other formats; the authors interpret their findings as capturing early-stage trends and technical directions. Looking ahead, they point toward small language models running locally, which could offer a feasible pathway to mitigating the digital divide in generative AI adoption, and toward multimodal systems that analyze handwritten responses and visual classroom contexts, extending generative AI beyond purely textual exchanges.
Subject of Research: A scoping review of empirical generative AI studies presented at leading educational technology conferences between 2023 and 2025.
Article Title: Generative artificial intelligence in education from a technology research perspective: a scoping review of empirical studies from leading educational technology conferences
Article References: Luo, J., Tang, Z., & Sun, M. (2026). Generative artificial intelligence in education from a technology research perspective: a scoping review of empirical studies from leading educational technology conferences. AI, Brain and Child, 2(1), Article 7. https://doi.org/10.1007/s44436-026-00031-9
Image Credits: AI Generated
DOI: 10.1007/s44436-026-00031-9
Keywords: generative AI, education, scoping review, large language models, AIED, learning analytics, K-12 education, automated grading, multi-agent systems, retrieval-augmented generation, educational data mining, small language models
News Source: Courtney Benton. (October 29, 2026). Scoping review maps how generative AI is reshaping education research. Scienmag.
