Fine-Grained Language Model Extracts Medication Data from Hungarian Clinical Narratives

Researchers at the University of Debrecen have developed and deployed a fine-grained, context-aware natural language processing system that extracts detailed medication information from unstructured Hungarian clinical notes with high precision and recall. Reporting in Frontiers in Artificial Intelligence, the investigative team—led by András Berzi alongside colleagues from the Institute of Medical Imaging, the University of Debrecen Clinical Centre, and the Department of Oncoradiology—evaluated named entity recognition models capable of distinguishing among distinct clinical contexts of drug use. The primary fine-grained context-aware model achieved an overall F1-score of 0.944 across five-fold cross-validation, while a complementary single-entity model reached an F1-score of 0.968. The resulting models have already been integrated directly into the university’s institutional data pipeline, where they are actively utilized to automate the extraction and structuring of medication records from more than 700,000 clinical documents annually.

The development of the extraction framework addresses a persistent challenge in health informatics regarding the completeness of medical records. While conventional structured electronic health records provide a reliable baseline for patient medication history, they frequently omit clinically vital contextual information that healthcare providers record exclusively within unstructured free-text clinical narratives. Critical elements such as unreported adverse drug events, patient-discontinued treatments, and over-the-counter supplement intake often reside entirely within narrative clinical notes. To unlock this information for clinical research and downstream analytics, the researchers designed an automated pipeline using the open-source spaCy natural language processing library, drawing source documents directly from the UDBD-Health Research Data Repository maintained by the University of Debrecen.

To capture the nuances of medication mentions, the researchers established a fine-grained annotation scheme that systematically divides medication entities into four distinct contextual categories. These categories comprise Current Medication (CURRENT_MED), Discontinued or No Medication (NO_MED), Adverse Events (AE_MED), and Allergies, designated under the CAVE classification (CAVE_MED). Preparing the corpus involved an initial automated pre-annotation phase powered by rule-based vocabularies, followed by rigorous manual refinement and curation performed by annotators within the Label Studio software environment. This structured contextual labeling allowed the extraction framework to differentiate between active pharmacological treatments, documented drug intolerances or allergies, and treatments that patients had ceased taking or were specifically advised to avoid.

To evaluate model generalization and secure a robust performance estimate free from data leakage, the investigators implemented a rigorous evaluation framework utilizing five-fold cross-validation with an inner validation split. Under this experimental design, model selection and early stopping criteria relied exclusively on an internal validation subset partitioned from within the training folds, while each outer held-out fold was strictly quarantined for final testing. In addition, the research team confronted a pronounced class imbalance across the four contextual classes, as mentions of adverse events and allergy-related medications appeared far less frequently than current prescriptions. To mitigate this imbalance, the team applied document-level oversampling to the minority-class training examples, creating an oversampled experimental model variation alongside the standard baseline architecture.

Model performance was systematically quantified using precision, recall, and F1-scores, reported as the mean plus or minus standard deviation across the five evaluation folds and benchmarked against a rule-based vocabulary-matching baseline. The standard fine-grained context-aware model (CME) achieved an overall F1-score of 0.944 ± 0.001. When evaluated across individual entity categories, CURRENT_MED demonstrated the strongest extraction performance with an F1-score of 0.958 ± 0.001. Mentions of allergies (CAVE_MED) yielded an F1-score of 0.900 ± 0.023, while discontinued or negated medications (NO_MED) registered an F1-score of 0.820 ± 0.005. The adverse events category (AE_MED) achieved an F1-score of 0.689 ± 0.026 and displayed the greatest degree of fold-to-fold variance among all evaluated classes. Applying document-level oversampling (CME OS) nudged the overall F1-score upward to 0.946 ± 0.002, delivering modest performance improvements for the underrepresented minority classes. In parallel, the coarse-grained single-entity model (ME) trained strictly for general medication recognition attained an F1-score of 0.968 ± 0.001 with minimal variance, substantially outperforming the rule-based baseline’s F1-score of 0.860 ± 0.003, which exhibited high precision but comparatively low recall.

By demonstrating the automated transformation of unstructured clinical text into structured pharmacological data, the study establishes an essential foundation for subsequent epidemiological investigations, real-world evidence research, and pharmacological safety studies in Hungarian healthcare settings. Nevertheless, the authors outlined several critical methodological limitations that warrant consideration. The training and evaluation data originated from a single clinical repository, raising questions regarding how well the models generalize across different healthcare institutions. Furthermore, adverse-event mentions remained underrepresented despite sampling adjustments, and the cross-validation protocol partitioned records at the document level rather than the individual patient level. The study also operated without independent parallel annotation, precluding the calculation of formal inter-annotator agreement metrics. Moving forward, the authors plan to expand the underlying text corpus, perform inter-institutional validation, and extend the extraction schema to capture granular clinical attributes including drug dosage and administration duration.

Subject of Research: Context-aware named entity recognition for medication extraction in Hungarian clinical narratives

Article Title: Granular by design: a fine-grained NER approach to context-aware medication extraction

Article References: Berzi, A., Berényi, E., Arató, V., Kallos-Balogh, P., Aranyi, S. C., Képes, Z., & Emri, M. (2026). Granular by design: a fine-grained NER approach to context-aware medication extraction. Frontiers in Artificial Intelligence, 9, Article 1927272. https://doi.org/10.3389/frai.2026.1927272

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1927272

Keywords: named entity recognition, natural language processing, electronic health records, medication extraction, Hungarian clinical narratives, spaCy, clinical informatics, machine learning, Granular, design, fine-grained, approach