Large language models running locally on hospital-grade hardware can produce clinical advice that sounds plausible, but a new study asks a harder question: would independent human experts actually accept the actions that such models recommend? A blinded assessment published in Discover Informatics put that question to eight multidisciplinary reviewers, who rated 400 synthetic clinical outputs generated by five locally deployed open-weight models. In a majority of cases the panel judged the primary action acceptable, yet one in seven outputs drew a majority verdict of unsafe or insufficiently urgent, and nearly every output that fell below a clinician-defined reference range was flagged as concerning.
The study was deliberately separated from an earlier technical benchmark by the same research group. That previous work, published in Cureus, examined prompt-induced variability and structured-output integrity using six synthetic acute-care cases and three local models. The new assessment instead generated a fresh set of 44 scenarios covering self-triage, counterfactual triage in base and worsened forms, synthetic reference-passage tasks, and contraindication or sequencing checks. No output records from the six-case study were reused, and the authors stress that the earlier benchmark supplied methodological motivation rather than the sampling frame for the present analysis.
Generation was extensive. The design called for 1584 candidate attempts, produced by crossing 44 scenarios with six source configurations, three prompt conditions and two repeated generations. One source configuration failed technically and was excluded before sampling, leaving 1320 eligible outputs across all 44 scenarios and five retained sources. From that pool, 400 distinct outputs were selected at random in Python, with no scenario subset, risk label, technical flag or expert rating used to steer the selection. The five retained sources were later unmasked for reporting as DeepSeek-R1 14B, Gemma 3 12B, gpt-oss 20B, Llama 3 and Gemma 4 26B, all run locally through Ollama. Model identity was withheld from raters during assessment.
Before any expert saw an output, an independent physician-researcher with more than 20 years of clinical experience defined lower and upper reference actions for each scenario. This person was separate from the eight output raters and supplied none of their ratings, and the ranges were never shown to the reviewers. The reference was a single-clinician judgment rather than a guideline-development panel or consensus gold standard, and the retained records do not establish whether the ranges were finalized before model-output generation. The authors treat agreement with these ranges as a secondary descriptive analysis, not a validation of a clinical standard.
Each of the eight reviewers rated all 400 outputs independently, producing 3200 ratings in total. The panel’s collective expertise spanned emergency medicine, critical care nursing, primary care, medication safety, medical informatics, infection control, internal medicine and patient-safety research. Reviewer-facing materials contained the synthetic scenario, any applicable synthetic source passage, and cleaned model advice including the primary action, urgency, rationale and diagnostic impressions. Model identity, prompt condition, risk labels, model self-evaluations and the clinician-defined action ranges were all withheld. Neither author rated any output, and no consensus adjudication was applied to the original ratings.
The headline result is that a majority of at least five of the eight reviewers judged the primary action acceptable for 338 of the 400 outputs, or 84.5 percent, with a Wilson 95 percent confidence interval of 80.6 to 87.7 percent. Sixty outputs drew a majority unsafe or insufficiently urgent verdict, 15.0 percent, and two outputs ended in four-four ties that remained in the denominator. Scenario-family bootstrap intervals, which resampled 36 scenario families while keeping paired base and worse scenarios and all eight ratings together, were 79.7 to 88.9 percent for acceptability and 10.8 to 19.7 percent for unsafe judgments. These intervals are conditional on the recruited panel and the synthetic scenario construction.
The most striking pattern emerged in the secondary concordance analysis. Sixty-one outputs fell below the archived lower reference action, 337 sat within the band, and two exceeded its upper boundary. Majority unsafe judgments occurred in 60 of the 61 below-range outputs, 98.4 percent, and in none of the 337 within-range outputs. Revision-stage sensitivity analyses reinforced the pattern: after excluding all eight synthetic reference-passage tasks, majority unsafe judgments appeared in 42 of 42 below-range outputs; after excluding only the three non-patient-triage information tasks, the figure was 54 of 55. The authors caution that these counts describe concordance with a clinician-defined scenario reference, not proof that the band is a unique or indisputable management plan, and no conclusion about over-escalation was drawn from the two above-range records.
Agreement statistics told a more nuanced story. Observed pairwise agreement on the primary action was 0.843, Fleiss’ kappa was 0.492 with a bootstrap interval of 0.416 to 0.554, and Gwet’s AC1 was 0.772 with an interval of 0.735 to 0.806. The authors report both coefficients because they rest on different chance-agreement models, and they warn that a larger AC1 should not be used to dismiss the lower kappa when rating marginals are imbalanced. For contraindication or sequencing problems, where only 11.4 percent of ratings were positive, kappa fell to 0.109 even though observed agreement was 0.820. Reviewer confidence, averaging 70.4 on a 100-point scale, showed almost no inter-rater consistency and was treated as exploratory.
Other questionnaire endpoints added texture without supporting a single safety claim. Major revision was endorsed by a majority for 44 outputs, 11.0 percent, and a contraindication or sequencing problem was reported in nine outputs, 2.3 percent. Adequate red-flag escalation received a majority positive judgment in 188 of 273 applicable outputs. Majority-positive counts were 329 for evidence support, 335 for overall clinical acceptability and 315 for patient-facing acceptability, but the authors insist these endpoints have different meanings and must not be combined into a claim that most outputs were clinically safe. Uncertain responses, which appeared only in the latter three items, were conservatively reclassified as negative in sensitivity analyses, lowering the corresponding positive rates.
The authors are explicit about what the study does not show. All scenarios and outputs were synthetic, selected by design rather than sampled from patients, and included explicit red-flag cues and fictional reference instructions labelled as such in the study materials. No patient outcomes, live clinician-system interactions or intervention benefits were measured, and expert-rated acceptability must not be read as a measured patient-safety outcome. The panel was recruited through one collaborating hospital, limiting generalizability to broader clinician populations, and the sensitivity analyses were added during revision rather than preregistered. Some historical technical details, including exact checkpoints, quantization settings and the original sampling script, were not retained, preventing exact reconstruction of the full generation workflow, although the archived 400-output rating analysis is reproducible from the supplied Zenodo archive.
What the findings do support, the authors argue, is a proposed tiered human-in-the-loop audit rather than a demonstration of governance effectiveness. One implementation would record generation failures and structured-output checks, sample usable outputs for blinded multidisciplinary assessment, and separately review action-range exceptions or uncertainty. Changes in a model, prompt, interface or serving stack would trigger a new audit rather than inherit evidence from a historical model name. The study also exposes an unresolved provenance problem: missing historical configuration fields illustrate how difficult end-to-end reconstruction remains. Future work, the authors suggest, should recruit multi-institutional panels, pre-document comparator development and test the proposed audit prospectively, treating clinician-in-the-loop review and prospective early-stage evaluation as complementary rather than interchangeable stages.
Subject of Research: Blinded expert assessment of the acceptability of primary actions recommended in synthetic clinical outputs generated by locally deployed large language models
Article Title: Blinded expert assessment of action acceptability in synthetic clinical outputs from local large language models
Article References: Harada, Y., & Kitazawa, T. (2026). Blinded expert assessment of action acceptability in synthetic clinical outputs from local large language models. Discover Informatics, 1(1), Article 34. https://doi.org/10.1007/s44564-026-00039-6
Image Credits: AI Generated
DOI: 10.1007/s44564-026-00039-6
Keywords: large language models, synthetic clinical scenarios, blinded expert assessment, human-in-the-loop audit, inter-rater agreement, reference-range concordance, local model deployment, clinical decision support, Fleiss kappa, Gwet AC1, medical informatics, patient safety
News Source: Denise Maddox. (November 5, 2026). Blinded experts flag unsafe actions in synthetic outputs from local clinical LLMs. Scienmag.
