Review maps how generative AI literacy is evidenced in language education

A systematic review of 27 empirical studies found that research on generative artificial intelligence literacy in English language education has most often directly evidenced strategic application and critical–ethical evaluation, while conceptual understanding was assessed less frequently. The review examined studies available from 2024 through March 2026 and organized the evidence under a provisional three-domain lens covering conceptual understanding, strategic application, and critical–ethical evaluation. Strategic application was directly evidenced in 21 studies, critical–ethical evaluation in 20, and conceptual understanding in 15. The distribution reflects differences in research and measurement emphasis, not evidence that teachers or learners possess stronger practical skills than conceptual knowledge.

The review followed PRISMA-informed procedures and searched Scopus, Web of Science, and ERIC. The searches identified 1,596 records: 611 from Scopus, 556 from Web of Science, and 429 from ERIC. After 491 duplicates were removed, 1,105 records were screened by title and abstract and 142 reports were assessed in full text. The final corpus comprised 27 peer-reviewed journal articles. Sixteen studies focused exclusively on higher-education language learners, while 11 centered on pre-service or in-service language teachers, teacher preparation, or professional learning. The literature was concentrated geographically in China, South Korea, and Hong Kong, with smaller groups from Türkiye, the United States, Iran, Saudi Arabia, Indonesia, and Vietnam.

Eligible studies involved generative AI or large-language-model applications in English language education and supplied empirical evidence relevant to generative AI literacy or adjacent capacities. The researchers excluded studies of non-generative AI, work outside language education, conceptual articles, reviews, conference abstracts or proceedings and book chapters without sufficient empirical detail, and reports lacking information needed for coding. Peer-reviewed journal articles were prioritized to support comparability and consistent reporting. Preprints, grey literature, conference proceedings, and technical AI venues were not included in the formal search, making the corpus a selective view of peer-reviewed research rather than a complete inventory of emerging work. The search endpoint also matters: the eight studies dated 2026 were available by the March 2026 search date and do not represent a complete publication year.

The synthesis explicitly matched each type of evidence to the claims it could support. Self-report instruments were treated as appropriate evidence of perceived literacy, including reported knowledge, awareness, confidence, readiness, or perceived competence, but not as automatic demonstrations of enacted ability. Task-based studies could support situated-use claims when learners or teachers were observed prompting, revising writing, using feedback, designing materials, or making pedagogical decisions. Researcher-designed prompts and comparisons of AI outputs were classified as indirect unless participants’ own understanding, use, evaluation, or decisions were observed. Twenty-two studies provided direct evidence for at least one of the three domains, while five provided indirect domain-relevant evidence without direct domain coding. This separation highlights a recurring constraint: proxies such as confidence, engagement, task performance, or improved writing products cannot automatically support comprehensive claims about literacy.

Strategic application was the most frequently evidenced domain, appearing in 21 of 27 studies. The activities coded under this domain included formulating and refining prompts, integrating AI feedback, revising English writing, creating lesson materials, planning instruction, and integrating generative AI into classroom interaction. Its visibility may partly reflect the relative ease of documenting observable tool use. By contrast, conceptual understanding concerns knowledge of generative mechanisms, affordances, limitations, probabilistic generation, hallucination, uncertainty, training-data constraints, and the status of AI-generated output. These qualities often require explanations or reflection rather than simple acknowledgment in a questionnaire. The review therefore describes differential emphasis in the literature and assessment practices, not a shortage of conceptual knowledge among language educators or learners.

Critical–ethical evaluation appeared directly in 20 studies and was nearly as prominent as strategic application. Studies addressed verification, responsible incorporation, academic integrity, authorship, corroboration of AI output, bias awareness, and judgments about whether generated content should be accepted, revised, attributed, or rejected. The depth of evaluation varied. Some studies placed checking and justification inside writing or teaching tasks, while others recorded general awareness of responsible use without observing how participants acted. Stronger operational evidence consisted of observable actions such as comparing a generated response with other information, identifying bias, explaining limitations, and justifying a decision to use, revise, or reject an output. These issues are particularly consequential in language education because generated material may function as language input, feedback, model text, revision advice, assessment support, or teaching material, with implications for voice, rhetorical appropriacy, cultural fit, and learner agency.

The analysis distinguished the breadth of a study from the coordination among its domains. Fourteen studies directly addressed all three domains, six addressed two, two were confined to a single domain, and five contained indirect evidence without direct domain coding. Among the 20 studies directly covering at least two domains, 18 showed substantive cross-component coordination and two showed only nominal integration. Substantive coordination meant that the represented domains were visibly connected across at least three of five components: aims or objectives, task or intervention design, instruments or evidence sources, results, and interpretation. This did not establish causal interaction among the domains or prove that they formed a deeply coherent theoretical system. Fourteen studies explicitly positioned literacy, competence, or a related construct as an instructional or assessment target, while 13 embedded literacy-related capacities within broader writing, feedback, lesson planning, critical-thinking, self-regulated-learning, or professional-development activities.

A review-specific appraisal rated four studies as low concern, 17 as having some concern, and six as high overall concern. Construct clarity, evidence appropriateness, and claim–measure alignment were treated as core domains, with separate consideration of relevance and methodological transparency. Six studies received high-concern ratings for construct clarity, and seven for claim–measure alignment. All five studies with only indirect domain evidence were rated high concern for domain-level literacy claims because their measures did not directly establish the targeted capacities; the reviewers clarified that this was not a judgment of their overall scholarly value. No research design or sample size was treated as inherently superior, so qualitative, case-study, small-sample, and self-report studies were not automatically penalized. In a calibration exercise, eight studies representing 29.6% of the corpus were independently double-coded, producing an overall Cohen’s kappa of 0.82 across domain presence, instructional positioning, and integration level. Disagreements were resolved through discussion, clarification of coding rules, and recoding where necessary.

The synthesis points toward an evidence agenda that distinguishes perceptions, situated use, task products, and comprehensive competence. Conceptual understanding could be examined through tasks asking participants to explain model affordances, limitations, uncertainty, hallucination, or appropriate conditions of use. Strategic application could be observed through prompt revision, feedback use, writing revision, materials design, and lesson planning. Critical–ethical evaluation could be assessed through source checking, bias identification, privacy judgments, authorship decisions, and justified acceptance or rejection of generated content. The review notes that think-aloud protocols, screen recordings, learning analytics, and learner reflections could connect process evidence with products and self-report, while longitudinal designs could examine change over time. It also identifies a need for comparisons across school levels, linguistic backgrounds, geographical regions, and resource conditions, because the retrieved evidence was concentrated in higher education, teacher education, and East Asian settings.

Within the search boundaries, generative AI literacy in English language education appears best characterized as situated and multidimensional, but the three-domain lens remains a provisional analytical tool rather than a validated final model. The central finding is not that any domain is inherently more important. It is that different evidence supports different strengths of claim: self-report captures perceived literacy, tasks capture situated use, products show task outcomes, and demonstrated multidomain literacy requires constructs, measures, evidence, and interpretations to be aligned. The review also identifies a conceptual boundary around prompting. Effective prompting can demonstrate strategic use, but additional evidence is needed to show that users understand generative systems and can judge reliability, bias, authorship, pedagogical value, and the consequences of incorporating generated content into language learning and teaching.

Subject of Research: How generative AI literacy is conceptualized, operationalized, and evidenced in English language education

Article Title: A systematic research synthesis of generative AI literacy in English language education

Article References: Zhang, Y., Qiu, X., & Wang, J. (2026). A systematic research synthesis of generative AI literacy in English language education. Discover Artificial Intelligence, 6(1), Article 1074. https://doi.org/10.1007/s44163-026-02178-z

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02178-z

Keywords: generative AI literacy, English language education, systematic review, AI in education, critical evaluation, prompting, teacher education, higher education, construct validity, research synthesis, systematic, research