AI-Structured Cancer CT Reports Win Over Readers in Prospective Survey Study

Radiology reports rewritten by a large language model into structured formats were rated more favorably than conventional free-text reports by a multidisciplinary group of readers, according to a prospective questionnaire study published in Cancer Imaging. Across 66 paired participants, structured oncologic CT reports showed significantly higher odds of agreement for 11 of 12 questionnaire items, with the largest gains in clinical usefulness, ease of finding information, clarity of recommendations, and suitability for patient communication.

The study, conducted at the Technical University of Munich between May and November 2025, used a two-phase, fixed-sequence within-participant design. Participants first rated five conventional free-text oncologic CT reports, one each for colorectal, breast, prostate, hepatocellular carcinoma, and lung malignancies. After a washout period with a median of 56 days, the same participants rated LLM-structured versions of the same reports generated by Llama-4-Scout-17B-16E-Instruct, deployed locally in an institutional Ollama environment with temperature set to zero.

The paired cohort comprised 24 medical students, 22 radiologists, 12 radiology technologists, and 8 oncologists, with a mean age of 30.5 years and 56.1 percent women. Phase 1 yielded 71 submissions; after one duplicate was excluded, four participants were lost to follow-up, an attrition rate of 5.7 percent. Participants rated each report on a 9-point Likert scale across 12 items organized into readability and comprehension, oncologic information quality, and clinical utility.

Clinical usefulness showed the highest odds ratio of all items, with mean ratings rising from 6.6 to 7.4 (odds ratio 4.81; 95 percent confidence interval 3.24 to 7.15). Findability of relevant information rose from 6.0 to 7.1 (OR 4.31), clarity of actionable recommendations reached an OR of 4.14, and suitability for patient communication an OR of 3.45. Structure and organization improved from 6.3 to 7.3 (OR 3.10), followed by completeness, precision, comprehensibility, and oncologic clarity. Readability alone showed no significant difference between formats (OR 1.15).

The analysis used generalized linear mixed models with a random intercept for participant, treating ratings of six or higher as agreement. Fixed effects included report format, expertise group, age, experience or training stage, and cancer domain. Reader background clearly influenced ratings: technologists showed significant improvements for all 12 items, students for 7, radiologists for 6, and oncologists for none. Oncologists gave relatively favorable baseline ratings to the free-text reports, particularly for completeness, recommendation clarity, and decision support, and the authors attribute their null result at least partly to the small subgroup of eight participants.

The authors highlight the inclusion of radiology technologists as a distinct reader group, noting that most structured reporting research has focused on radiologists and referring clinicians. They suggest the strong improvements among technologists may reflect the value of predictable report organization for readers less practiced at navigating free-text narrative variability, while radiologists’ more moderate gains may reflect greater familiarity with narrative reports. Prior survey data cited in the paper show radiologists more often prefer unstructured reporting than non-radiologists do.

The structured templates followed a common oncologic framework covering clinical history, imaging technique, comparison with prior imaging, TNM-relevant findings, additional findings, and impression, mapped to the eighth edition of the AJCC TNM classification and LI-RADS v2018. Information absent from the source report was labeled N/A rather than inferred. Before the survey, each structured report was compared with its source by two radiologists, with disagreements reconciled by a senior board-certified radiologist. The authors stress this was author-led quality control of survey materials, not a formal quantitative accuracy study.

That distinction carries substantial weight in the paper’s interpretation. The authors explicitly caution that higher reader ratings for completeness and precision should not be interpreted as evidence that the LLM conversion was free of hallucinations, omissions, unsupported additions, altered measurements, or laterality or site errors. Previous studies of LLM-based report conversion have reported residual omissions, confabulations, and clinically relevant misclassifications, with performance varying across models, prompts, templates, languages, and tasks. The findings, the authors write, support the perceived value of radiologist-reviewed LLM-structured reports but do not establish the safety of autonomous report conversion.

Several limitations constrain the conclusions. The single-center convenience sample may reflect local reporting practices, and no external participant completed both phases. Because all participants read free-text reports first, format was confounded with study period; the washout may have reduced direct recall but cannot exclude familiarity, learning, expectancy, or other temporal effects, so the results should be read as preferences under a fixed sequence rather than an order-independent causal effect of format. Participants could not be blinded to the visibly different formats, novelty and demand effects cannot be excluded, and only one synthetic report per cancer domain was evaluated, limiting generalizability across staging, restaging, and surveillance scenarios. The inclusion of a prostate cancer CT report assessed presentation of information in that report and does not imply CT suffices for all prostate cancer assessment.

The authors conclude that formal source-to-output validation and prospectively planned, randomized multicenter assessment, including reading time, clinical decision-making, and patient-relevant outcomes, remain necessary before clinical implementation of LLM-assisted report structuring.

Subject of Research: Reader preferences for large language model-structured versus free-text oncologic CT reports

Article Title: Multidisciplinary attitudes towards LLM-structured oncologic CT reports: a prospective fixed-sequence questionnaire study

Article References: Busch, F., Marka, A. W., Wingberg, L. F., Prucker, P., Kim, S. H., Strotzer, Q., Schmidt, C., Lammert, J., Borm, K. J., Peeken, J., Makowski, M. R., Adams, L. C., & Bressem, K. K. (2026). Multidisciplinary attitudes towards LLM-structured oncologic CT reports: a prospective fixed-sequence questionnaire study. Cancer Imaging, 26(1), Article 140. https://doi.org/10.1186/s40644-026-01137-2

Image Credits: AI Generated

DOI: 10.1186/s40644-026-01137-2

Keywords: large language models, structured reporting, oncologic CT, radiology, questionnaire study, clinical utility, report readability, radiologists, radiology technologists, oncologists, medical students, Cancer Imaging