Building reliable artificial-intelligence training data in Thai has long presented a problem that is easy to underestimate. Large language models can generate fluent questions and answers from documents, but fluency does not guarantee that a question is meaningful, that its answer is supported by the source, or that the wording reflects how Thai is actually used. A new study describes a framework designed to address those weaknesses by turning dataset construction into a controlled, auditable workflow rather than a single automated generation step. Called the Adaptive Prompt Feedback Loop, or APFL, the system produced 118 accepted question-and-answer pairs from 150 document chunks, retaining 78.67 percent of the material that entered its evaluation pipeline. In comparison, the tested zero-shot language-model baselines retained between just two and 61 pairs under the same quality gates. The researchers say the result is not evidence that artificial intelligence can replace expert annotation, but that carefully constrained AI systems can shift human labour toward the examples most in need of scrutiny.
The challenge is particularly acute for Thai, one of many languages that receive less computational attention than English. Thai writing does not conventionally separate words with spaces in the same way as English, making segmentation an important preprocessing step for search and language modelling. Long-form documents also contain headings, references, tables, legal or technical terminology and passages whose meaning depends on context. If such material is divided poorly, a retrieval system may return incomplete evidence or combine unrelated passages. A question-generation model may then produce an answer that sounds plausible but cannot be justified from the document. These errors are often described as hallucinations, but in a dataset they become more than isolated mistakes: they can be absorbed into the examples used to train and evaluate future systems. The APFL researchers therefore treated document preparation, evidence retrieval, generation, evaluation and human review as connected stages, each with the potential to block defective output.
At the heart of APFL is retrieval-augmented generation, or RAG. In a conventional language-model interaction, a model generates text from patterns learned during training and from the prompt it receives. RAG adds an information-retrieval stage: relevant passages are first selected from a document collection, and those passages are supplied as evidence for the model’s response. Technically, this usually involves converting text into numerical representations called embeddings, which place semantically similar passages near one another in a high-dimensional vector space. A query can then be compared with those vectors to identify candidate evidence. The model is instructed to generate a question and its answer from the retrieved material, rather than relying solely on its internal knowledge. APFL held retrieval and evaluation settings fixed while allowing prompts to be refined in response to failures. By keeping those underlying settings stable, the researchers aimed to distinguish improvements in prompt policy from changes caused by shifting retrieval parameters or evaluation criteria.
The adaptive loop is designed to be conservative. When a generated item fails a quality check, the system does not regenerate the entire dataset indiscriminately. Instead, it records the failure, updates the prompt-level guidance and attempts regeneration only for the failed item. Once an example passes the required gates, it is “frozen” and retained rather than repeatedly rewritten. This fail-only regeneration limits unnecessary computation and reduces the risk that an already acceptable question will be degraded by further model variation. Items that continue to fail are sent through a residual-routing process for human review. In this arrangement, the model functions as a component within a workflow that has explicit stopping rules and an audit trail. The feedback loop can therefore be examined after the fact: researchers can see which outputs failed, what kind of defect was detected, how the prompt policy changed and whether the revised item eventually met the criteria.
The evaluation used a controlled subset of 150 chunks drawn from three cross-domain collections of Thai long-form documents. The source areas were enterprise risk management and environmental, social and governance material; Thailand’s digital-government planning; and a guide to the use of herbal medicines in medical practice. This combination exposed the system to administrative, policy and health-related language rather than a single narrow topic. The study compared APFL with strong zero-shot large language-model baselines, meaning the comparison systems were asked to perform the task without task-specific examples in their prompts. Crucially, the models were assessed using identical rubric-based quality gates, and the evaluation was implemented separately from the generation process. That separation matters because a model should not be allowed to define its own success criteria without an independent check. The gates examined whether questions were answerable from the evidence, whether answers were faithful to the source, and whether the resulting pairs were usable as dataset entries.
APFL’s 118 accepted pairs represented a substantial improvement over the baselines in this controlled test, but the headline retention rate requires context. The experiment began with only 150 chunks, so it was not a massive benchmark covering the full range of Thai writing, dialects or real-world user queries. The authors also did not claim that every accepted item was flawless. Instead, they conducted a domain-balanced human audit of 90 sampled APFL outputs. In that audit, 97.78 percent of the examples were judged practically viable, meaning they required no correction or only minor human edits. “Practically viable” is a deliberately operational measure: an item may be suitable for use after a small wording adjustment even if it is not ready to be published without review. The finding suggests that the framework reduced the burden of creating examples from scratch, but it does not eliminate the need for people who understand Thai, the subject matter and the intended use of the dataset.
That distinction is especially important for the health-related documents in the test. A question-and-answer pair about herbal medicines can be grammatically correct and apparently grounded while still being misleading if it strips away a warning, confuses a preparation with an active ingredient or implies a clinical recommendation that the source does not make. Similar problems can occur in government or governance material, where a changed qualifier—such as “may,” “must,” or a specific implementation period—can alter the meaning of a policy statement. APFL’s emphasis on evidence-span selection is intended to make such claims traceable to particular sections of a document. Human auditors can then verify not only whether the answer resembles the source, but whether the selected evidence genuinely supports it. This is a stronger requirement than generic language quality, because a polished answer can still be factually unsupported. The framework’s human-overseen design acknowledges that semantic and domain-level judgments remain difficult to automate reliably.
The study also frames APFL as a cost-aware approach to annotation. Fully manual construction of question-answer datasets requires an expert to read source material, identify important facts, write a question, formulate an answer and check the pair for accuracy. Automation can accelerate those steps, but uncontrolled automation creates a second workload: people must discover and repair large numbers of subtle errors. APFL attempts to alter that balance by having models generate candidate pairs and perform initial checks, while directing human attention to frozen outputs that need no intervention only minimally and to residual cases that repeatedly fail. The operational benefit is not simply a reduction in model calls. It is the strategic allocation of scarce expert time. Instead of reviewing every item with equal intensity, reviewers can concentrate on ambiguity, unsupported claims, retrieval mistakes and domain-sensitive language. The authors describe this as a move from full manual annotation toward targeted audit, validation and residual correction.
The approach nevertheless has limitations that temper its viral appeal. Its results come from a relatively small, controlled sample and from one language and document setting. Performance could change with noisier scans, informal Thai, regional variants, highly specialized terminology or questions requiring information spread across multiple passages. The comparison also does not establish that APFL will outperform every modern model or every alternative annotation pipeline. Automated rubrics can themselves miss errors, and language-model evaluators may share some of the biases or blind spots of the generators they assess. The generated research datasets were not publicly released because they contain derived annotations, evaluation records and workflow information tied to source documents, although the authors say access may be granted upon reasonable request subject to institutional policies and copyright constraints. Independent replication will therefore be important, particularly with larger samples, publicly inspectable evaluation records and audits by reviewers who are not involved in developing the framework.
What APFL offers is less a magic prompt than an engineering philosophy for low-resource language AI. It treats dataset construction as a sequence of quality-controlled decisions, combines retrieval with explicit grounding, preserves successful outputs, regenerates only failures and keeps humans responsible for the cases machines cannot settle. The 78.67 percent acceptance rate and the 97.78 percent practical-viability rate in the sampled audit suggest that this combination can make Thai question-generation and question-answering datasets more scalable without pretending that automation is infallible. As language models spread into education, public administration, healthcare information and search, the quality of the data behind them will increasingly shape whether they are useful or dangerously convincing. For Thai and other under-resourced languages, the study’s broader message is that trustworthy progress may depend not on removing people from the loop, but on designing systems that show people where their judgment matters most.
Subject of Research: Human-overseen artificial-intelligence framework for constructing Thai question-generation and question-answering datasets from long-form documents
Subject of Research: Technology and Engineering
Article Title: APFL: A controlled agentic framework for Thai QG–QA dataset construction via adaptive prompt feedback loop
Article References: APFL: A controlled agentic framework for Thai QG–QA dataset construction via adaptive prompt feedback loop — https://doi.org/10.1007/s12652-026-05114-2 Original publication
Image Credits: AI Generated
DOI: 10.1007/s12652-026-05114-2
Keywords: Thai natural language processing, question generation, question answering, retrieval-augmented generation, human–AI evaluation, dataset construction, prompt feedback, language-model auditing
Tags: Adaptive Prompt Feedback Loop (APFL) frameworkAdaptive Prompt Feedback Loop (APFL) framework for question generationaddressing fluency versus meaningfulness in Thai question-answer pairsaddressing Thai language segmentation issuesAI training data generation for Thai languagechallenges in Thai natural language processingchallenges of Thai language processing in AI datasetscomparison of AI and human annotation in Thai dataset creationcomparison of AI-generated versus human-annotated datacontrolled dataset construction workflowscontrolled workflow for dataset construction in Thai NLPcreating reliable Thai question-answer datasetsenhancing quality control in Thai language datasetsenhancing reliability of Thai QG-QA datasets with AIimproving quality of AI-generated Thai questions and answersimproving question relevance and answer support in Thai QA systemslanguage-specific challenges in large language model fine-tuninglanguage-specific issues in Thai text segmentationleveraging AI to improve Thai NLP applicationsreducing human labor in Thai NLP dataset annotationreducing manual annotation effort with AI
