AI chatbots show promise and peril in epilepsy care, major review finds

Generative artificial intelligence chatbots such as ChatGPT are being tested across nearly every corner of epilepsy medicine, from reading discharge summaries to flagging patients who may have been missed for epilepsy surgery, but a new comprehensive review concludes that the technology is not yet ready to be trusted on its own. The analysis, published in Acta Epileptologica, systematically mapped 24 studies of modern generative large language models in epilepsy-related clinical tasks and found a field that is advancing quickly in narrow, text-focused jobs while remaining strikingly immature everywhere else.

The review, conducted by Shihao Ge, Yueqian Sun and Qun Wang, followed the PRISMA-ScR framework and searched PubMed, Embase and the Web of Science Core Collection from inception to May 2026. Of 529 records identified, 334 remained after deduplication, and 24 studies survived full-text screening. To be included, a study had to use a generative large language model or generative foundation model as the primary analytical or assistive engine in an epilepsy-specific clinical or clinically oriented task. Older encoder-style clinical natural language processing models, conventional machine-learning classifiers and seizure-prediction signal-modeling frameworks were deliberately excluded from the core synthesis and used only as contextual comparators.

The clearest signal emerged in structured information extraction from unstructured clinical text. A fine-tuned GPT-4 pipeline achieved an F1 score of 85.8 percent for extracting seizure frequencies from narrative notes, outperforming several BERT-based and generative comparator models. A separate study of clinic letters found that open-source models could pull out epilepsy type, seizure type, current antiseizure medications and associated symptoms more effectively than a conventional clinical NLP comparator, although implicit and highly specialized information remained harder to capture. In seizure-outcome extraction, a prompt-engineered Llama2 model reached 74 percent median accuracy on epileptologist notes, far above zero-shot BERT at 25 percent but below a fully fine-tuned BERT benchmark of 84 percent; on sparser general-neurology notes, the best prompt reached 76 percent and actually exceeded the fine-tuned BERT model’s 67 percent.

One 2026 single-center cost-effectiveness study extended this extraction paradigm to real-world surgical-outcome surveillance. A Mistral 24B pipeline extracted Engel outcomes from electronic health record notes with 95.65 percent extraction accuracy, supported 91.2 percent binary good-versus-bad outcome classification, and generated an incremental cost-effectiveness ratio estimate that differed by only 2.08 percent from the prospective-database approach. The review’s authors stress that these performances were obtained within carefully defined extraction workflows and should be read as evidence for task-bounded information structuring rather than broader clinical reasoning. They do not establish safe unsupervised clinical use.

Diagnostic tasks told a more cautionary story. In a retrospective emergency-department study applying the 2014 ILAE practical definition of epilepsy, ChatGPT-4 diagnosed epilepsy less often than neurologists and showed essentially absent agreement, with very low sensitivity despite moderate specificity. In a prospective simulation of 37 adult vignettes derived from real seizure presentations, ChatGPT-4 showed high sensitivity for identifying epileptic seizures and diagnosing epilepsy, at 96.9 percent or better, but poor specificity for diagnosing epilepsy at just 26.7 percent and for distinguishing acute symptomatic from unprovoked seizures at 33.3 percent; human experts consistently performed better. Notably, the models could recognize well-defined syndromes yet often failed at the more basic step of diagnosing epilepsy before attempting syndrome classification.

Efforts to distinguish epileptic seizures from functional or dissociative seizures were similarly mixed. On patient narratives, GPT-4’s balanced accuracy improved from 57 percent in the zero-shot setting to 64 percent with one-shot prompting, without further gains from additional examples, and remained below the 71 percent achieved by experienced neurologists. In another study, GPT-4 received information sequentially, first history, then EEG, then neuroimaging, and under a modified referral-oriented analysis achieved 85 percent accuracy and an AUC of 87 percent, outperforming ChatGPT and a supervised functional-seizure score. Yet agreement with the structured score was only fair, and repeated runs changed predictions in 33 percent of patients, meaning favorable discrimination did not translate into stable decisions. A small prospective multimodal study of 24 smartphone-recorded events from 15 patients found that Gemini models achieved only 25.0 to 54.2 percent accuracy in distinguishing epileptic from functional seizures without additional clinical context, despite high self-rated confidence.

Presurgical applications showed some of the most striking results, with important caveats. In 16 single-center surgical cases, investigators converted an expert-built epilepsy ontology into a JSON knowledge base for a customized GPT matcher using retrieval-augmented generation; adding EEG text improved accuracy from 75.0 to 87.5 percent for the ontology system and from 87.5 to 93.8 percent for the GPT/JSON matcher, though the integrated system was evaluated without ablation, so the improvement cannot be attributed to retrieval-augmented generation alone. With semiology alone, ChatGPT-4 showed regional sensitivity of 80 to 90 percent for frontal and temporal localizations but only 20 to 40 percent for parietal, occipital and insular regions and just 3 percent for the cingulate cortex. Multisource specialist reports yielded 98.3 to 100 percent laterality accuracy in 154 seizure-free surgical patients from two centers, yet lobar performance remained model- and subtype-dependent, and insular localization remained poorest. Because semiology reflects the symptomatogenic zone rather than the epileptogenic zone itself, these predictions did not necessarily identify the true seizure origin.

Perhaps the most immediately consequential finding concerned surgical referral screening. In a retrospective tertiary-clinic study using raw Hebrew clinical notes from 110 patients, GPT-5, GPT-5 mini, o4-mini and Gemini models identified epilepsy-surgery eligibility with sensitivity up to 1.00 and specificity up to 0.96, and majority voting captured all eligible patients in that cohort. Among patients who met surgical criteria, 45 percent had not previously been considered for surgery. A separate implementation-style study over a 12-month period combined a random-forest screen flagging the top 5 percent of 310 clinic patients with a Mistral-7B-Instruct model that collated referral-relevant information from the chart; among 15 manually reviewed high-risk cases, 8 met prespecified criteria for epilepsy-surgery evaluation but only 3 were referred within one month. Extraction accuracy for selected structured elements ranged from 80 to 100 percent, but performance was weaker for management-plan summarization, and the model produced hallucination, omission and copy-forward errors. Both studies support LLM-based referral flagging under clinician review, not autonomous surgical triage.

Patient-facing uses raised the sharpest safety concerns. When tested on ketogenic diet planning for children with epilepsy, ChatGPT, Copilot and Gemini all produced unsafe or inaccurate menus, with major errors in macronutrient targets, ketogenic ratios and food suitability across classic ketogenic, medium-chain triglyceride and modified Atkins diets, even though the outputs read fluently. In an earlier benchmark, ChatGPT-3.5 produced correct and comprehensive responses in only about two-thirds of cases, with prognosis-related answers performing particularly poorly. In emergency-oriented testing, ChatGPT recognized benzodiazepines as first-line therapy for status epilepticus but omitted or inconsistently reported several guideline elements unless prompts were made more specific, and misleading recommendations were still observed. A seizure-first-aid counselling experiment found that baseline GPT-4o failed to assess whether seizures were ongoing and whether the person was conscious or breathing, whereas an expert-customized GPT-4o improved checklist fulfillment from 14 to 49 percent to 77 to 92 percent but still required simulation and human validation. In prognosis, a single-center retrospective cohort of adults with status epilepticus found ChatGPT-4o achieved 75.8 percent sensitivity but only 36.6 percent specificity for discharge survival, showed no advantage over the established STESS score, and produced unstable results across two runs of the same standardized prompt.

The review’s authors conclude that current evidence supports a limited role for cautious, clinician-supervised use of generative LLMs in selected text-heavy epilepsy tasks, particularly extraction, summarization, documentation support and patient education, but falls short of justifying independent or routine clinical deployment. The evidence base was heterogeneous, with frequent reliance on retrospective or simulated designs, single-center data and limited external validation, alongside persistent concerns about hallucination, bias, model opacity and workflow integration. The review itself had limitations: it was a scoping review without meta-analysis or formal evidence grading, restricted to English-language full-text studies in three databases with no grey-literature search, and it deliberately excluded the broader spectrum of AI methods used in epilepsy. The authors call for future studies with narrower task definitions, epilepsy-specific datasets, transparent reporting of model versions and prompt design, external validation, and explicit documentation of clinician verification workflows. Until then, they write, general-purpose LLMs should not be used for epilepsy diagnosis or treatment decisions without clinician review, and the most defensible near-term uses remain drafting, extraction, summarization and research support with source checking and explicit clinician sign-off.

Subject of Research: Applications, limitations and future directions of generative large language models in epilepsy care

Article Title: Modern generative large language models in epilepsy care: a scoping review of current applications, challenges, and future directions

Article References: Ge, S., Sun, Y., & Wang, Q. (2026). Modern generative large language models in epilepsy care: a scoping review of current applications, challenges, and future directions. Acta Epileptologica, 8(1), Article 32. https://doi.org/10.1186/s42494-026-00271-5

Image Credits: AI Generated

DOI: 10.1186/s42494-026-00271-5

Keywords: large language models, epilepsy, artificial intelligence, scoping review, clinical decision support, ChatGPT, information extraction, patient safety, hallucination, epilepsy surgery, seizure diagnosis, clinician supervision