LiveCtx Boosts Conversational AI Accuracy by Selecting Context at Query Time

In an evaluation encompassing five hundred long-duration benchmark dialogues, computer scientists Yu Liu and Chuanshuang Wang demonstrated that deferring context reduction until an exact question is submitted significantly improves answer accuracy in conversational artificial intelligence systems. Reporting in Discover Computing on an adaptive framework named LiveCtx, the researchers found that their query-time assembly policy attained 79.8 percent and 81.8 percent accuracy across two independent automated evaluators under a standardized client working budget of 131,072 characters. The approach departs fundamentally from conventional memory designs that compress or truncate conversational records as dialogues progress. Instead of permanently rewriting prior interactions using static rules unconditioned on future queries, the system preserves original session transcripts intact within persistent storage and evaluates overall context length only after receiving an incoming prompt, thereby avoiding premature information loss while adhering to client resource boundaries.

Managing long conversational histories presents an enduring technical challenge for autonomous agents deployed across multi-session interactions. Dialogue records routinely accumulate across hundreds of distinct sessions, expanding serialized transcripts beyond the operational context limits of large language models or client application constraints. While contemporary foundation models increasingly support expanded input windows, simply enlarging token capacities does not resolve which specific historical interactions are relevant to answering a prompt. Historical data may contain subtle state changes, chronological dates, exceptions, or user preferences that seem negligible when initially uttered but later become critical supporting evidence. Standard conversational systems typically address this length constraint by enforcing rolling sliding windows that preserve only contiguous recent turns or by generating periodic recursive summaries that rewrite earlier exchanges. Both write-time strategies permanently discard fine-grained conversational nuances before the relevance criteria of subsequent inquiries can be known.

To overcome the limitations of write-time summarization, the LiveCtx architecture implements a store-retrieve-assemble pipeline that cleanly separates persistent multi-turn history from temporary, query-specific working contexts. Complete dialogue sessions are stored indefinitely alongside chronological metadata, stable identifiers, and dense semantic vector representations. Upon receiving a user prompt, the query analysis module extracts temporal expressions, identifies explicit entity mentions, and computes dense text embeddings using the BGE-Large-EN-v1.5 model. Stored sessions are scored using a linear scoring function that combines semantic similarity, literal temporal matches, and substring entity overlap using fixed heuristic weights of 0.5, 0.2, and 0.3, respectively. The system selects an initial candidate pool of up to fifty sessions, generates deterministic two-thousand-code-point previews, and submits them to an automated reranking model. The top ten ranked sessions are subsequently reconstructed in full from persistent storage for final context assembly.

The central innovation within the framework is its Adaptive assembly policy, which enforces a strict zero-or-one generative compression rule based on explicit length checks. Rather than performing incremental token pruning across individual dialogue turns, the policy evaluates the serialized character count of the complete candidate context against the client working limit. If the entire unpruned dialogue history fits within the assigned character threshold, the system passes the full conversational record directly to the generator, bypassing query analysis, retrieval, reranking, and summarization entirely. If the full history exceeds the budget but the ten restored complete sessions fit within the threshold, those retrieved sessions are supplied to the generation model completely uncompressed. Only when the restored retrieved set also surpasses the working budget does LiveCtx execute a single, query-conditioned generative compression pass across the entire set, producing a temporary, non-destructive summary that never overwrites stored history.

The researchers systematically assessed the LiveCtx framework on the LongMemEval-S benchmark, a curated dataset comprising five hundred evaluation questions designed to test long-term conversational memory. The dataset includes 470 answerable queries spanning single-session user and assistant facts, multi-session information synthesis, temporal relationships, user preference tracking, and dynamic knowledge updates, as well as 30 unanswerable queries intended to measure model hallucination and refusal behavior. Experimental generation was conducted using the Qwen3-Max model via the DashScope API, configured with zero temperature, disabled thinking tokens, and non-streaming output under a client-side limit of 131,072 Unicode scalar values. To ensure rigorous evaluation standards, all outputs were independently graded by two advanced judge models, DeepSeek-v4-pro and GLM-5.2, with failed API requests and malformed outputs counted strictly as incorrect responses within the fixed denominator of five hundred instances.

Under the primary 131,072-character budget, the LiveCtx-Adaptive configuration achieved top-tier performance, reaching 79.8 percent accuracy under DeepSeek-v4-pro and 81.8 percent under GLM-5.2. In paired bootstrap comparisons, Adaptive exceeded LiveCtx-Filter—an identical retrieval baseline that retains complete retrieved sessions without generative compression—by 4.4 and 5.0 percentage points across the two grading models. Against standard vanilla retrieval-augmented generation relying purely on semantic similarity, Adaptive demonstrated an accuracy advantage of 7.2 and 7.6 percentage points. Adding large language model reranking to semantic retrieval yielded modest, statistically nonsignificant improvements of 1.4 and 1.0 percentage points over vanilla retrieval. Meanwhile, LiveCtx-PerSession, an alternative assembly variant that compresses individual sessions separately whenever they cross remaining capacity thresholds, attained between 73.0 and 77.0 percent accuracy, while sliding windows and rolling summarization baselines achieved markedly lower performance.

Detailed tracking of execution paths across varying budget constraints demonstrated that the performance advantage of the Adaptive policy is concentrated within over-budget retrieval scenarios. Because all five hundred LongMemEval-S conversation histories exceeded the tested budgets of 96K, 128K, and 160K characters, the full-history bypass was never activated in the primary benchmark. At the restrictive 96K threshold, the retrieved session set exceeded capacity in 495 cases, requiring Adaptive to execute its single compression routine on 99 percent of runs and yielding a 4.2 to 4.4 percentage point gain over Filter. At 128K, compression was triggered on 342 queries, where the median context length for Adaptive was approximately 407 characters compared to 131,072 characters for Filter. At 160K, however, 472 retrieved session sets fit within the limit uncompressed, reducing compression invocations to 28 queries and shrinking the performance gap between Adaptive and Filter to a nonsignificant 0.0 to 0.8 percentage points.

An analysis across diverse question types revealed that the benefits of adaptive query-time compression were most pronounced on complex multi-hop inquiries. When evaluated under the two automated graders, Adaptive correctly answered 10 and 11 more multi-session synthesis questions and 15 and 17 more temporal reasoning questions than the uncompressed Filter baseline. In contrast, performance differences remained minimal on single-session factual queries and knowledge updates. Furthermore, the evaluation uncovered a key behavioral limitation regarding unanswerable queries. On the thirty benchmark questions requiring refusal, Adaptive properly declined to answer only 22 times under joint evaluator agreement, achieving an abstention rate of 73.3 percent. This trailed Filter and PerSession at 24 refusals, retrieval with reranking at 25, rolling summaries at 27, and recent contiguous windows at 29, demonstrating that highly concentrated contextual summaries can inadvertently promote unwarranted answering confidence.

To scrutinize the factual integrity of context reduction, the authors conducted a detailed evidence-retention study across a fifty-question subset, tracking compression frequency alongside target fragment preservation. While Adaptive successfully decreased generative compression calls by 42.2 percent relative to PerSession—invoking 37 total compression calls compared to 64—it retained fewer official target fragments. Evaluators found that PerSession retained 115 of 117 official target fragments, achieving a 98.3 percent retention rate, whereas Adaptive preserved 104 and 106 fragments across the two judges. The investigators noted that this retention discrepancy stems partly from cumulative output limits, as Adaptive restricted whole-set compression to a single 2,048-token generation, whereas PerSession permitted up to 1,024 tokens for each separately compressed session. In subsequent blinded majority-vote panels, the two methods exhibited no statistically significant difference in overall answer support, indicating that reduced fragment counts did not diminish immediate query resolution.

The researchers also examined framework generalization on the LoCoMo benchmark, evaluating 1,986 questions across ten extended conversational interactions where every dialogue history naturally fit within the 131,072-character limit. Under these conditions, LiveCtx-Adaptive executed its first architectural pathway, providing uncompressed, complete historical transcripts directly to the generator without initiating retrieval or summarization. In conversation-macro evaluations, this complete-history approach surpassed retrieval with reranking by 0.776 percentage points in lexical F1 and 3.481 percentage points in category-aware F1. Conversely, executing unrestricted full-history prompts on LongMemEval-S outside client limits yielded 69.0 percent and 69.8 percent accuracy despite consuming an average of 123,868 provider prompt tokens. This outcome confirmed that presenting massive, uncurated conversational records directly to language models introduces contextual distraction, underperforming targeted query-time context selection even when models possess sufficient nominal context window capacity.

Despite its demonstrated advantages, the authors documented several technical constraints and design trade-offs within the LiveCtx framework. End-to-end ablation experiments removing the query analysis module or the large language model reranker produced slight numerical drops in accuracy that failed to reach statistical significance following Holm correction, indicating that individual component contributions warrant further investigation. Additionally, the initial scoring weights for semantic, temporal, and entity signals were derived heuristically from a small, non-independent pilot sample. The authors emphasized that the benchmark relied exclusively on automated neural evaluators rather than blinded human judges, and that client-side character budgets do not capture operational latency, billing costs, or cache behavior. Future developments will focus on integrating post-retrieval evidence sufficiency checks to improve abstention accuracy on unanswerable questions and validating query-time assembly across diverse conversational domains.

Subject of Research: Query-time context selection and adaptive assembly for long conversational histories

Article Title: LiveCtx context selection at query time for long conversational histories

Article References: Liu, Y., & Wang, C. (2026). LiveCtx context selection at query time for long conversational histories. Discover Computing, 29(1), Article 855. https://doi.org/10.1007/s10791-026-10807-5

Image Credits: AI Generated

DOI: 10.1007/s10791-026-10807-5

Keywords: artificial intelligence, conversational agents, context management, retrieval augmented generation, natural language processing, information retrieval, machine learning, LongMemEval, LiveCtx, context, selection, query