Agentic Framework Refines Free-Text Chest X-Ray Reports Using Small Language Models

In a multi-institutional investigation covering ninety-seven thousand examinations across ten clinical sites, structured radiology supervision improved vision-language model performance for chest X-ray report generation over conventional free-text training. Publishing in the International Journal of Computer Assisted Radiology and Surgery, researchers Md Badhon Miah, Lukas Buess, and Andreas Maier found that organizing chest radiography findings into standardized anatomical sections enabled Qwen2.5-VL-3B to achieve superior clinical report quality across all ten participating evaluation sites, supported by ninety-five percent confidence intervals derived from fifty thousand bootstrap resamples. To address the critical preprocessing challenge of converting unstructured clinical narratives into structured formats without relying on commercial cloud interfaces, the investigators introduced RadStructAgent, an agentic framework designed to structure free-text reports using open-source small language models. Tested on one thousand clinical reports, the framework utilized iterative orchestration, multi-stage judge feedback, and sequential refinement to boost clinical accuracy and anatomical placement, demonstrating that agentic verification can significantly enhance local open-source models for medical dataset curation.

Chest radiography represents one of the most common imaging examinations in routine medical practice, making automated medical report generation a prominent research target to alleviate radiologist workload and improve clinical workflow efficiency. However, training multimodal vision-language models on clinical reports is persistently hindered by the unstructured, highly variable nature of radiology documentation. Narrative reports vary substantially across hospitals in terminology, style, sentence structure, reporting preferences, and anatomical organization. When vision-language models are trained with unstructured free-text supervision, they must dedicate representational capacity to arbitrary linguistic variation rather than diagnostic visual-pathological observations, while anatomy-level quality assessment remains exceedingly difficult. Structured Radiology Report Generation directly mitigates this issue by organizing narrative findings into predefined anatomical compartments, providing a consistent supervision signal during multimodal training and enabling anatomy-aware evaluation of generated diagnostic outputs.

Despite the recognized benefits of structured supervision, assembling structured training datasets at scale remains difficult because most public chest X-ray repositories are available only in unstructured free-text format. Prior benchmarking showed that while proprietary large language models like Gemini achieve high structuring performance, open-source small language models struggle under standard single-pass inference settings. In single-step generation, small models frequently omit crucial diagnostic findings, generate unsupported hallucinations, misassign statements to incorrect anatomical sections, or introduce redundant entries across sections. Nevertheless, proprietary cloud models are often impractical for healthcare institutions due to operating costs, privacy regulations, institutional data governance constraints, and prohibitions against transmitting sensitive health records to external servers. Furthermore, many clinical centers operate under constrained computing budgets. These limitations create an acute practical need for reliable, resource-efficient methods that perform structured report conversion locally using open-source small language models.

To meet this need, the authors designed RadStructAgent, a model-agnostic agentic framework that converts unstructured findings into standardized anatomical categories without external cloud dependencies. Given an input findings section, a base small language model structurer generates an initial structured report candidate using a dedicated structuring prompt that specifies eight predefined anatomical sections: Lungs and Airways; Pleura; Cardiovascular; Hila and Mediastinum; Tubes, Catheters, and Support Devices; Musculoskeletal and Chest Wall; Abdominal; and Other. This initial candidate is stored in candidate memory, and the subsequent iterative refinement loop is managed by a persistent shared state. At each discrete step, the shared state tracks the active candidate, findings-level feedback, anatomy-level feedback, verification status flags indicating whether each judge has evaluated the current candidate, the candidate history, the tool-call count, the completed revision count, and the remaining execution budget, ensuring complete operational traceability.

An orchestrator powered by the same underlying language model serves as the controller, evaluating the shared state to select an action from a defined five-part action space. This operational action space comprises findings judging, anatomy judging, sequential revision, candidate selection from memory, and final output termination. The orchestrator returns a JavaScript Object Notation decision detailing its chosen action, reasoning, and certainty score. If the orchestrator outputs an invalid action, malformed syntax, or a decision that breaches preconfigured budget boundaries, conservative fallback rules handle the exception by continuing candidate validation, selecting the best available candidate from historical memory, or finalizing the active report. By enforcing structured outputs and deterministic fallback logic, the orchestrator systematically coordinates multi-step error correction while preventing uncontrolled looping during automated report conversion.

RadStructAgent uses complementary prompt-guided tool modules that target distinct structuring failure modes. The findings judge evaluates the structured candidate against the source free-text report, outputting structured feedback on missing findings and unsupported findings. Missing findings are true clinical observations present in the source text but omitted from the candidate, whereas unsupported findings represent model-generated hallucinations lacking textual justification in the original report. The anatomy judge evaluates anatomical placement, flagging wrong-section allocations where findings appear under improper anatomical headings, along with duplicate statements appearing across multiple sections. When revision is selected, the framework applies a two-step sequential revision pipeline rather than attempting unified multi-objective revision. A findings revision agent first updates the candidate using source text and findings feedback to resolve missing and unsupported observations, after which an anatomy revision agent refines the intermediate draft using anatomy feedback to correct misclassifications and eliminate redundancies, storing the revised candidate in memory.

The framework integrates candidate selection and explicit stopping safeguards to ensure inference remains computationally bounded and cost-effective. When candidate selection is invoked, a dedicated selector module reviews all historical candidates stored in memory alongside the original free-text report. The selector chooses the candidate that best aligns with the source narrative, evaluating source-supported content, anatomical placement accuracy, duplicate content minimization, and linguistic conciseness, falling back to the current candidate if selection fails. The refinement cycle halts automatically when the orchestrator finalizes the candidate, when the selector retrieves the optimal report, or when revision yields an empty or unchanged candidate. Crucially, strict operational caps restrict execution to a maximum of twenty tool calls and ten revision rounds per report, preventing unnecessary computational refinement while preserving predictable processing bounds for local offline workflows.

The experimental validation assessed both downstream vision-language model fine-tuning and upstream report structuring under standardized local execution conditions. Downstream multimodal modeling used ReXGradient-160K, an open multi-institutional chest radiograph dataset with paired images and reports. For report generation, ninety-seven thousand samples across ten institutions were split independently within each institution into eighty percent training, ten percent validation, and ten percent testing sets, providing a combined test set of nine thousand six hundred and fifty-six examinations. Structuring experiments evaluated one thousand findings reports sampled across institutions from the established Structured Radiology Report Generation Benchmark. RadStructAgent was executed on a single NVIDIA A100 graphics processing unit with forty gigabytes of memory, serving model inference through vLLM with four-bit bitsandbytes quantization, an OpenAI-compatible local interface, a thirty-two thousand seven hundred and sixty-eight token context window, zero point nine zero memory utilization, single-worker execution, and deterministic decoding with temperature set to zero.

The empirical findings confirmed the efficacy of structured supervision for multimodal clinical report generation. When fine-tuned on chest radiographs using structured supervision, Qwen2.5-VL-3B outperformed free-text training across the combined ten-institution test set. Clinical report quality measured by the primary metric GREEN rose from 0.494 to 0.539, disease classification measured by F1-SRR-BERT increased from 0.489 to 0.533, and CRIMSON advanced from 0.447 to 0.457, while embedding-level RaTEScore remained unchanged at 0.608. Site-level analysis confirmed that structured supervision improved GREEN scores across all ten individual hospital sites. In the structuring task, RadStructAgent improved open-source small language models over single-pass baselines. For Qwen3-4B, RadStructAgent increased GREEN from 0.844 to 0.867 and anatomical placement accuracy on RadFAA from 0.883 to 0.891. For Qwen3-8B, GREEN increased from 0.863 to 0.881 and RadFAA rose from 0.889 to 0.902, demonstrating that agentic refinement narrows the gap between open-source models and proprietary references.

Execution analysis revealed that RadStructAgent adapts refinement dynamically to individual reports. Across the one-thousand-report evaluation set, judge-approved finalization occurred in eighty-seven percent of Qwen3-4B cases and eighty-one percent of Qwen3-8B cases, with remaining cases terminating via fallback paths like budget limits or unchanged revisions. This iterative refinement introduced an acceptable runtime trade-off: average inference time per report increased from 0.7 seconds to 6.7 seconds for Qwen3-4B, and from 1.0 second to 13.2 seconds for Qwen3-8B. Component build-up ablations showed that blind revision alone left GREEN unchanged while raising RadFAA by 0.008 for Qwen3-4B and 0.015 for Qwen3-8B. Adding findings-judge feedback improved GREEN by 0.023 for Qwen3-4B and 0.017 for Qwen3-8B, while adding anatomy judging finalized overall gains at 0.023 GREEN and 0.008 RadFAA for Qwen3-4B, and 0.018 GREEN and 0.013 RadFAA for Qwen3-8B. Model-family testing on Gemma-4-E4B-it confirmed generalizability, raising GREEN from 0.914 to 0.923 and RadFAA from 0.924 to 0.934. The authors noted limitations including reliance on automated metrics, potential judge hallucinations, and a single anatomy judge, recommending future investigations into radiologist validation, prompt sensitivity, and multi-model role allocation across agent functions.

Subject of Research: Agentic structuring of chest X-ray reports with small language models

Article Title: RadStructAgent: agentic structuring of free-text chest X-ray reports with small language models

Article References: Miah, M. B., Buess, L., & Maier, A. (2026). RadStructAgent: agentic structuring of free-text chest X-ray reports with small language models. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03811-z

Image Credits: AI Generated

DOI: 10.1007/s11548-026-03811-z

Keywords: radiology report generation, vision-language models, small language models, chest X-ray, agentic AI, structured reporting, medical AI, clinical NLP, RadStructAgent, agentic, structuring, free-text