The study, published in Discover Artificial Intelligence, presents a prototype that combines local large language models with a curated corpus of historical documents to provide grounded answers without relying on cloud-based services. The primary objective was to address the limitations of general-purpose large language models, which often hallucinate or lack factual grounding when dealing with low-resource languages and specialized historical domains. By restricting the system to a single, authoritative 62-page document on Karnataka heritage sites, the team aimed to create a controlled environment for testing the feasibility of offline, language-specific knowledge retrieval.
The core challenge identified by the authors is the significant asymmetry in multilingual representation within pre-trained language models. Kannada, like other Dravidian languages, occupies a very small fraction of the web-scale corpora used to train modern AI systems. This scarcity leads to degraded semantic embeddings, meaning that queries in Kannada may not retrieve relevant passages effectively. Furthermore, the morphological richness of Kannada, characterized by agglutinative structures and complex conjunct consonants, complicates token-based matching and standard chunking methods. The research team noted that without specific adaptations, English-optimized RAG systems are unreliable for retrieving heritage knowledge in this language, often failing to capture the nuances of classical Kannada and Sanskrit registers found in historical inscriptions and architectural descriptions.
To overcome these technical hurdles, the proposed pipeline integrates several specialized components. The ingestion process uses a hybrid strategy involving PyMuPDF for native text extraction and Tesseract OCR for scanned pages, with a specific Kannada language model to handle complex ligatures. A crucial step in the preprocessing phase is Unicode NFC normalization, which ensures that variant Unicode code points for the same Kannada grapheme are treated uniformly. This is essential because different software or OCR engines may produce different code point sequences for identical characters, which would otherwise cause semantically equivalent tokens to be treated as distinct inputs by the embedding model, thereby reducing retrieval recall.
The system employs recursive text chunking based on a Kannada-aware separator hierarchy. Instead of fixed-size character splitting, which can break factual statements, the algorithm prioritizes paragraph boundaries, then line boundaries, and finally sentence separators, including both Latin periods and Kannada danda markers. Each chunk is assigned a 50-character overlap to reduce the likelihood of losing information that spans chunk boundaries. Source metadata, including the filename and page number, is inherited by all child chunks, allowing the system to provide documentary provenance for every generated response. This traceability allows responses to be linked back to specific source documents and pages, which is important for heritage and educational applications where correctness and authenticity are essential.
For retrieval, the system uses the paraphrase-multilingual-mpnet-base-v2 sentence encoder to transform document chunks into dense vectors. Although Kannada is not among the languages used for the model’s multilingual alignment training, it is represented through the underlying XLM-RoBERTa encoder. The vectors are indexed in a FAISS exact nearest-neighbor index, where inner product is equivalent to cosine similarity for unit vectors. A deterministic passage selection rule governs which chunks are supplied to the generator. A candidate chunk is retained only if its similarity score meets both an absolute floor of 0.55 and a relative floor of 0.85 times the maximum score. This dual-threshold approach ensures that the number of retrieved passages is computed from the score distribution rather than selected manually, and it allows the system to abstain from answering if no passage clears the absolute floor, thereby preventing the generation of unsupported claims.
The generation phase utilizes locally deployed large language models via the Ollama framework, enabling fully offline operation. Five model configurations were tested: llama3, llama3:8b, mistral:7b, gemma3:4b, and phi3. All models were run with identical decoding settings, including a temperature of 0.3 and a maximum output length of 150 tokens, to ensure that comparisons reflected model performance rather than sampling configuration differences. The prompts were constructed entirely in Kannada to enforce monolingual response generation and discourage code-switching into English, a common behavior in multilingual models when processing low-resource language queries. Each prompt explicitly instructed the model to answer only from the provided context and to state if the information was unavailable.
The evaluation was conducted on a set of 25 Kannada questions authored by the research team, stratified into three difficulty tiers. The Easy tier consisted of 8 factual recall questions answerable from a single passage, the Medium tier included 9 interpretive questions requiring synthesis across two or three passages, and the Hard tier comprised 8 analytical questions demanding comparative analysis across multiple sites and dynasties. The questions were drawn from the content of the single 62-page heritage document used as the corpus. The response quality was measured using four metrics: ROUGE-1 F1, ROUGE-L F1, alignment-based scoring, and BERTScore F1. These metrics were computed using a Kannada-aware tokeniser that normalizes text to NFC and segments Kannada, Latin, and numeral runs separately to preserve content such as dates and inscription identifiers.
The results indicated that gemma3:4b achieved the highest scores across all response-quality metrics and the lowest mean latency. On the aggregate set, gemma3:4b recorded a BERT-F1 of 0.855, with tier-specific scores of 0.850 for Easy, 0.863 for Medium, and 0.851 for Hard questions. Its mean latency was 3.97 seconds, significantly faster than llama3 (17.46 seconds) and llama3:8b (12.98 seconds). In contrast, phi3 produced very long responses averaging 325 words with a latency of 93.78 seconds, suggesting verbose generation that did not align with the concise answers required. The study noted that while gemma3:4b led on surface metrics, its BERT-F1 margin over the next best model, mistral:7b, was 0.019, which is within the estimated run-to-run variability of the pipeline. No significance testing was performed, and the results are descriptive point estimates that do not support inferences about how the models would rank on other questions or documents.
A significant finding from the evaluation was the divergence between contextual and surface overlap metrics. BERT-F1 remained high and nearly flat across all difficulty tiers for every model, varying by at most 0.024 between a model’s best and worst tier. However, ROUGE and alignment scores fell sharply on the Medium and Hard tiers, reaching near-zero for the llama3 variants. This pattern suggests two possible interpretations: either BERTScore correctly captures semantic equivalence that surface metrics miss in a morphologically rich language, or BERTScore has a compressed dynamic range in Kannada, assigning high similarity to any fluent text on a related topic. The authors argue that neither family of metrics is self-validating without retrieval-level and human evaluation, as response metrics alone cannot distinguish between a correct answer and a fluent but factually incorrect one.
The study acknowledges several limitations that constrain the generalizability of its findings. The evaluation was performed on a single curated document and a small question set written by the developers, which may favor the system in ways that independent analysis cannot correct. Retrieval quality was not measured, as the questions lacked annotations for the specific evidence needed to answer them. Consequently, it is unknown whether the high BERT-F1 scores reflect effective retrieval or merely the model’s ability to generate fluent text on the correct topic. Additionally, the retrieval encoder was not trained with Kannada parallel data, and its performance for Kannada-specific semantic fidelity remains untested. The system also lacks a mechanism to detect conflicts between retrieved passages, a limitation that would become critical in a multi-source corpus.
The system’s ability to operate without network access makes it a candidate for deployment in rural heritage tourism corridors where cloud connectivity is unreliable. The authors suggest that future work should focus on measuring retrieval quality and grounding on independently annotated question sets, validating metrics against expert human judgement, and expanding the corpus to include epigraphic records and archaeological reports. They also propose the integration of Indic-specific language models and hybrid lexical-dense retrieval to improve the handling of proper nouns and dates, which are crucial for accurate heritage interpretation.
Subject of Research: A Kannada centric retrieval augmented generation system for Karnataka heritage knowledge access
Article Title: A Kannada centric retrieval augmented generation system for Karnataka heritage knowledge access
Article References: G. P., R., B. M., A., M, S. N., D, D., & N., P. (2026). A Kannada centric retrieval augmented generation system for Karnataka heritage knowledge access. Discover Artificial Intelligence, 6(1), Article 1762. https://doi.org/10.1007/s44163-026-02641-x
Image Credits: AI Generated
DOI: 10.1007/s44163-026-02641-x
Keywords: Kannada, centric, retrieval, augmented, generation, system, Karnataka, heritage, knowledge, access, scientific research
News Source: Denise Maddox. (November 10, 2026). Kannada Heritage RAG System Tested on 25 Questions. Scienmag.
