Study Evaluates Small Language Models for Plain-Language Simplification of Medical Reports

A research team from the Universidad Carlos III de Madrid has evaluated several artificial intelligence language models to determine their capability to simplify medical reports for non-expert readers. Published in the journal Scientific Reports, the study addresses a widespread challenge in healthcare communication, where patients often struggle to understand technical summaries of their clinical conditions. By translating dense clinical phrasing into accessible plain language, the initiative aims to help individuals better understand their diagnoses and adhere more effectively to prescribed medical treatments.

According to study co-author Paloma Martínez, a Full Professor in the university’s Department of Computer Science, data indicate that 46 percent of European citizens encounter significant difficulties when trying to comprehend medical documentation. To address these communication barriers, researchers designed tools that enable language models to generate dual versions of clinical documents: one retaining complete technical detail for healthcare practitioners and a companion version tailored specifically to help patients interpret specialized abbreviations and complex phrasing.

The study aligns with emerging legal and regulatory requirements for cognitive accessibility across public services. In Spain, Royal Decree 707/2026, which was published in the Official State Gazette on September 3, will come into force on January 2, 2027, establishing the first regulatory framework governing basic cognitive accessibility conditions. This decree sets binding standards to ensure that administrative procedures, public documents, and essential services are understandable for citizens, carrying direct statutory implications for public administrations operating in healthcare, justice, employment, transport, and digital administration.

The methodological foundation of the simplification process rests on established international standards for accessible communication. Co-author Lourdes Moreno, also a Full Professor of Computer Science at the institution, notes that plain language functions as a standardized methodological framework designed to ensure information is easy to find, understand, and use. Rather than merely shortening sentences or substituting individual words, the approach requires structuring content logically, adapting technical vocabulary, and explicitly addressing the primary informational needs of non-specialist readers.

To evaluate model capabilities empirically, the researchers assembled a corpus of 50 clinical reports provided by the Andalusian Society of Family and Community Medicine. Each original report underwent three independent adaptations developed by cognitive accessibility specialists adhering to the international plain language standard ISO 24495-1:2023, generating a benchmark dataset of 150 adapted reference texts. These paired source documents and expert adaptations provided the primary training and testing material used to fine-tune and assess the selected computational models.

The researchers conducted comparative evaluations across four distinct language model families: Llama 3.2 at parameter sizes of 1B and 3B, Salamandra at 2B and 7B, RigoChat-v2 at 7B, and Medical-mT5 at 3B. Across these systems, the authors measured simplification effectiveness, semantic similarity, factual consistency, text readability, and environmental footprint. The findings demonstrated that no single model proved superior across every dimension: Llama-3.2-3B exhibited the most balanced overall profile regarding computational efficiency, operational robustness, and general performance, whereas RigoChat-v2-7B achieved higher scores when prioritizing readability and raw output quality.

Detailed qualitative and quantitative assessments of generated texts identified three persistent error types present across all four evaluated model families: oversimplification, clinical hallucinations, and textual incoherence. Oversimplification involved the accidental deletion of vital clinical facts necessary for patient care; hallucinations entailed fabricating or distorting data found in the original source records; and incoherence manifested as fragmented syntax or abrupt transitions that undermined text comprehension. Because of these recurrent generation errors, the authors emphasized that direct human clinical oversight remains essential before deploying language models in sensitive medical settings.

To encourage reproducible evaluation and further collaborative development, the research team made their complete experimental assets available to the broader scientific community. Conducted within the HumanAI project under Spain’s State Plan for Scientific, Technical and Innovation Research and supported by the State Research Agency, the project’s fine-tuned models, evaluation prompts, and dataset have been published as open-source resources in public repositories including GitHub and Zenodo.

Subject of Research: Evaluation of small language models for clinical text simplification into plain language

Article Title: New AI tools to simplify medical reports and enhance doctor-patient communication

Article References: New AI tools to simplify medical reports and enhance doctor-patient communication. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: artificial intelligence, clinical reports, plain language, health literacy, language models, cognitive accessibility, medical informatics, open science, tools, simplify, medical, reports