Large language models can write fluent, confident prose that is completely wrong, and the hardest errors to catch are the ones that read beautifully. A new study published in Discover Artificial Intelligence presents a low-resource framework for detecting exactly this kind of failure, known as fluent overgeneration hallucination, where a model's output is grammatically polished but contains excessive, false, or misleading information that is not supported by the input. The work, by Anirud Ramani and Manju Venugopalan, tackles a persistent bottleneck in AI safety research: there is far more unlabeled text available than human-verified examples of hallucination, and manual annotation is slow, expensive, and subjective.
The researchers built their system on the SHROOM dataset released for SemEval-2024 Task 6, a shared task dedicated to hallucination detection. The corpus covers three natural language generation tasks: machine translation, definition modeling, and paraphrase generation. It contains a pool of 60,000 unlabeled training examples alongside 4,000 expert-annotated samples divided into validation and test sets.
After cleaning, removing duplicates, and guarding against data leakage between splits, the team worked with 58,649 unlabeled training instances, 1,000 labeled validation instances, and 2,995 labeled test instances. The labeled data showed a slight imbalance toward the non-hallucinated class, a detail that would shape many later design decisions. The results were sobering for the heavier models: the best traditional setup, a random forest with TF-IDF features, reached an accuracy of 0.6230 with a Spearman correlation of 0.1910, essentially matching the deep learning models.
The authors suggest that when labeled data is scarce and hallucinations often hinge on surface-level mismatches, shallow models with simple features can be surprisingly competitive. Transformer models told a different story. Fine-tuned versions of BERT Large, ELECTRA Large, RoBERTa Large MNLI, and DeBERTa v3 Large were evaluated, and DeBERTa v3 Large emerged as the strongest overall, achieving 76.56 percent accuracy and a Spearman correlation of 0.6277.
The Spearman metric matters here because it measures whether the model's confidence scores rank in agreement with the proportion of expert annotators who judged each output as hallucinated, capturing the spectrum of severity rather than a simple yes-or-no verdict. Multi-seed averaging confirmed that these results were stable, with low standard deviations across runs. The authors found that stricter thresholds produced cleaner pseudo-labels, while relaxed cutoffs introduced noisy labels and unstable training.
Pseudo-labeling alone had a serious weakness: it favored the majority class. An analysis of class distribution across self-training iterations showed that hallucinated samples made up only 7.4 percent of the pseudo-labeled data in the second iteration and never exceeded 30 percent, even as the model maintained stable accuracy. A second annotator independently reviewed all 150 manually labeled samples, achieving 85.3 percent agreement with the first annotator, corresponding to a Cohen's kappa of 0.70, which the authors describe as substantial agreement.
Disagreements clustered on borderline cases involving partial factual inconsistencies or ambiguous overgeneration, underscoring how subjective this task remains even for humans. The finalized configuration achieved an accuracy of 0.78 and a Spearman correlation of 0.6626 on the held-out test set, with bootstrap confidence intervals of 0.7713 to 0.8000 for accuracy and 0.6501 to 0.6896 for the correlation, significant at p below 0.01. For context, the official SHROOM baseline achieved 0.697 accuracy and 0.403 Spearman correlation on the model-agnostic track.
Zero-shot baselines based on FactCC and natural language inference performed poorly, highlighting that general-purpose consistency tools transfer awkwardly to this task setting. Generalization beyond the training distribution proved harder. When the final ensemble was evaluated without retraining on roughly 4,500 human-annotated ChatGPT responses from the HaluEval dataset, accuracy fell to 0.5878, with a strong F1-score of 0.7151 on non-hallucinated instances but a weak 0.2544 on hallucinated ones.
The authors attribute this drop to differences in generation style and annotation assumptions between the datasets. A sensitivity analysis showed the framework was robust to moderate changes in the number of pseudo-labels added per class, and a qualitative review found the model handles clear semantic errors well but sometimes penalizes acceptable rewording or compression, and occasionally misses subtle additions that shift meaning. The study's contribution is deliberately modest and practical: it demonstrates that established semi-supervised techniques, carefully combined, can deliver competitive hallucination detection with only about 1,150 hand-labeled examples and no proprietary or massive models.
The authors are candid about limitations, noting that their thresholds and sampling strategies were tuned for this specific setting, that the reported results correspond to a representative execution of the pipeline, and that a comprehensive multi-seed evaluation of the complete framework remains future work. As language models are deployed ever more widely in high-stakes domains, lightweight and reproducible detection methods of this kind offer a pragmatic path toward catching fluent fabrications before they reach the public.
Subject of Research: Low-resource detection of fluent overgeneration hallucinations in large language model outputs across machine translation, definition modeling, and paraphrase generation tasks
Article Title: Low-resource hallucination detection in LLMs on multi-task datasets via iterative pseudo-labeling using confidence thresholding and active learning
Article References: Ramani, A., & Venugopalan, M. (2026). Low-resource hallucination detection in LLMs on multi-task datasets via iterative pseudo-labeling using confidence thresholding and active learning. Discover Artificial Intelligence, 6(1), Article 1011. https://doi.org/10.1007/s44163-026-02106-1
Image Credits: AI Generated
DOI: 10.1007/s44163-026-02106-1
Keywords: hallucination detection, large language models, semi-supervised learning, pseudo-labeling, active learning, Query by Committee, DeBERTa, SemEval-2024 SHROOM, natural language generation, machine translation, paraphrase generation, definition modeling
News Source: Denise Maddox. (October 27, 2026). Research report: Low-resource hallucination detection in LLMs on multi-task datasets via iterative pseudo-labeling using confidence thresholding and active learning. Scienmag.
