How VS Code Copilot Chat Compacts Your Conversation

VS Code Copilot Chat uses a compaction mechanism to summarize older conversation history when the agent session exceeds the LLM's context window, triggered automatically or via the /compact command. The process relies on the budget-aware prompt renderer @vscode/prompt-tsx, which prunes low-priority content and escalates to compaction when pruning fails.

LLMs have limited context windows. When an agent session grows too long, VS Code Copilot Chat uses compaction to summarize older history while keeping recent work verbatim. I traced exactly how it works from the source. Note The standalone microsoft/vscode-copilot-chat repo was archived in May 2026 . The agent code now lives in the main microsoft/vscode https://github.com/microsoft/vscode repo under extensions/copilot/ , and it has changed since the standalone repo froze. Everything below is traced against the current extensions/copilot source. Source files all under extensions/copilot/ : src/extension/intents/node/agentIntent.ts — budget math, trigger logic src/extension/prompts/node/agent/summarizedConversationHistory.tsx — prompt, LLM call, history selection, re-insertion src/extension/prompts/node/agent/backgroundSummarizer.ts — the async state machine and thresholds src/extension/prompts/node/agent/simpleSummarizedHistoryPrompt.tsx — the Simple-mode fallback Note Microsoft moves fast. Treat line numbers as approximate and the thresholds as true the day I looked. First, what is @vscode/prompt-tsx ? Compaction makes no sense until you understand the renderer underneath it. @vscode/prompt-tsx https://github.com/microsoft/vscode-prompt-tsx the repo pins ^0.4.0-alpha.8 is a budget-aware prompt renderer . Instead of concatenating strings into a messages array, you author the entire prompt as a tree of TSX components — a PromptElement — and the renderer flattens it into the ChatMessage that goes to the model, fitting it to the token budget for you . Here’s the actual agent prompt, lightly trimmed agentPrompt.tsx : return < {baseInstructions} <AgentConversationHistory flexGrow={1} priority={700} promptContext={ctx} / <AgentUserMessage flexGrow={2} priority={900} {...userMessageProps} / <ChatToolCalls flexGrow={2} priority={899} toolCallRounds={ctx.toolCallRounds} truncateAt={maxToolResultLength} / </ ; Two numbers on each element do the work: — when the rendered tree is over budget, the renderer drops the priority lowest-priority content first. The current user message 900 and the latest tool calls 899 outrank older history 700 , which outranks boilerplate system instructions. Same-priority siblings are pruned in declaration order.— controls how flexGrow leftover budget is distributed. A flexGrow element renders after its siblings and receives whatever budget they didn’t use. So the user message and tool calls flexGrow={2} get first claim on the window; conversation history flexGrow={1} fills the remainder. Every element’s render state, sizing is handed a PromptSizing with tokenBudget and an async countTokens , so an element can measure itself and trim to fit. That’s literally how tool results cap themselves at 50% of the window truncateAt={maxToolResultLength} above , and there are helpers for the common cases: <TokenLimit max={2000} {/ hard cap on a subtree /}</TokenLimit <PrioritizedList priority={p} descending={false} {rounds}</PrioritizedList TokenLimit caps a subtree; PrioritizedList hands a list of children descending or ascending priorities — exactly how conversation rounds are ranked so the oldest get pruned first; TextChunk keeps as much of a long string as fits. Pruning is graceful — until it isn’t. When even the highest-priority content won’t fit, prompt-tsx throws a BudgetExceededError . That exception is the signal the agent loop catches to escalate from “just prune” to “compact the history” — which is where the rest of this post begins. Overview Compaction is the escalation when prompt-tsx’s pruning isn’t enough. There are two automatic triggers — both on by default now — plus the user-initiated /compact command: | Path | Trigger | Default? | |---|---|---| Foreground summarize | BudgetExceededError while rendering hard overflow | ✅ on | Background compaction | post-render context ≥ ~80% jittered 0.78–0.82 with a warm cache; ≥ 0.90 cold-cache emergency | ✅ on | Manual /compact | user runs the command | — | Note This is the biggest change since the standalone repo. The old build gated background compaction behind a chat.backgroundCompaction experiment and had a separate ≥85% “proactive inline” experiment. Both flags are gone. Background compaction now ships on by default gated only by the master summarization switch , and the inline mechanism it used became how background compaction works rather than a separate path. When it triggers The budget is computed from config and the effective context size, then trimmed for tools: js const baseBudget = Math.min summarizeThresholdTokens ?? effectiveMaxTokens, effectiveMaxTokens // clamp: never above the real max ; // 10% safety margin on the message portion when tools are present const messageBudget = Math.max 1, Math.floor baseBudget - toolTokens 0.9 ; Two subtleties: effectiveMaxTokens honors the user’s Context Size picker not just the model’s raw modelMaxPromptTokens , and the threshold config is interpreted as either a ratio 0–1 , a fraction of the window or an absolute token count ≥100 — the ambiguous gap in between throws. Drag the fill level to see which path fires: Foreground is the safety net — it catches the hard overflow and recovers: } catch e { if e instanceof BudgetExceededError && summarizationEnabled { result = await renderWithSummarization budget exceeded ${e.message} ; } } Background compaction exists to avoid the cost of “render, fail, re-render.” After each turn renders, it checks the post-render context ratio and, if you’re over the threshold, kicks off summarization in the background so the compacted result is ready to apply on the next render. Two details make it cache-friendly: Cache-warmth gating + jitter. It only fires at the normal ~ 0.80 threshold when the prompt cache is warm a completed tool-call round this turn , and it jitters the exact trigger across 0.78–0.82 so it doesn’t always fire at the same boundary. With a cold cache it waits for the ≥0.90 emergency line. Thresholds live in BackgroundSummarizationThresholds . Apply-min ratio. A finished background summary is discarded if the context ratio has since dropped below 0.65 — e.g. you switched to a larger-context model and no longer need it. Mechanically, background compaction doesn’t make a separate “summarize this” request. It folds a “compact now” instruction into a forked copy of the same render for prompt-cache parity and parses a <summary block back out of the model’s reply. How it works Pick the cut point. Render history in reverse newest first , keeping recent rounds verbatim. Exclude the overflowing round. With multiple tool rounds, drop the last one — it’s what pushed over the limit. Stop at the previous summary. Walking back, break at the first round that already has a .summary . Compaction compounds instead of re-summarizing from scratch. Generate the summary. Call the LLM with the structured format below. Re-insert. Wrap the summary in a <conversation-summary user message that replaces the older turns; store it as turn metadata so it survives the next turn. // Excluding the round that blew the budget: if toolCallRounds && toolCallRounds.length 1 { toolCallRounds = toolCallRounds.slice 0, -1 ; // last round overflowed summarizedToolCallRoundId = toolCallRounds.at -1 .id; // summarize from the prior one } What gets kept vs summarized Recent rounds → kept verbatim, so the model picks up mid-task. The round that overflowed → excluded from the summary. Everything before the previous summary → already represented; not re-included. Tool results → truncated at maxToolResultLength , which the agent loop sets to 50% of the model’s max prompt tokens modelMaxPromptTokens 0.5 , keeping the head and tail of long outputs a 40/60 split and dropping the middle. The flat 2000 you may see in the source is a different feature — the panel’s chat-summary renderer — not in-loop compaction. The summary format The summarization is a real LLM call on the same model the conversation uses , at temperature: 0 streaming is intentionally not disabled — there’s even a regression test asserting the request doesn’t force stream: false . The prompt doesn’t ask for “a summary” — it demands an 8-section handoff document, emitted inside a <summary block after a separate <analysis block: 1. Conversation Overview — objectives, session context, intent evolution 2. Technical Foundation — core tech, frameworks, architectural patterns 3. Codebase Status — each file touched: purpose, state, key code 4. Problem Resolution — issues hit, solutions, debugging context 5. Progress Tracking — done vs. partially done vs. validated 6. Active Work State — exactly what was being worked on last 7. Recent Operations — last commands, tool results, pre-summary state 8. Continuation Plan — pending tasks and the immediate next step There are two modes: Full sends tool definitions with tool choice: 'none' and Simple . If Full fails, it falls back to Simple. The breadcrumb Compaction is lossy — the verbatim history is gone. So the summary carries a pointer to the full transcript on disk: summary += \nIf you need specific details from before compaction such as exact code snippets, error messages, tool results, or content you previously generated , use the ${ToolName.ReadFile} tool to look up the full uncompacted conversation transcript at: "${transcriptPath}" ; // ...then appends the transcript's current line count and an example call This breadcrumb is added whenever the session has an on-disk transcript path no experiment flag — that gate existed in the old build but is gone now . It’s appended exactly once, at summary-creation time, and baked into the frozen summary text — so later renders replay it verbatim, preserving Anthropic’s prompt cache. The summary is the fast path; the transcript file is the escape hatch the model reads only when it needs an exact detail. Same instinct as progressive disclosure /posts/stop-bloating-your-claude-md-progressive-disclosure-ai-coding-tools/ Stop Bloating Your CLAUDE.md: Progressive Disclosure for AI Coding Tools AI coding tools are stateless—every session starts fresh. The solution isn't cramming everything into CLAUDE.md, but building a layered context system where learnings accumulate in docs and specialized agents load on-demand. : cheap index in context, expensive detail on demand. Model-specific gotchas | Model | Handling | |---|---| Opus claude-opus | Extra instruction: do not call tools, only write text. | Anthropic + thinking | Last thinking block preserved and re-attached as the first thinking block after the summary. | Anthropic + tool search | Client-side tool search tool-use/result pairs stripped before the call, or Anthropic 400s. | Gemini | Orphaned function call s whose results got pruned stripped, or it 400s. | GPT-4.1 | A “keep going” reminder is appended after the summary. | Prompt caching | Summary “baked” once so later renders don’t bust Anthropic’s cache. | PreCompact hook | Fires before summarizing; can archive the transcript. Errors never block. | Settings | Config key | Default | What it does | |---|---|---| chat.summarizeAgentConversationHistory.enabled | on | Master switch — gates both foreground and background compaction. | chat.advanced.summarizeAgentConversationHistoryThreshold | model max | Lower the budget that triggers compaction ratio 0–1 or absolute tokens ≥100 . | chat.advanced.agentHistorySummarizationMode | auto | Force simple or full summary mode. | chat.conversationCompaction.usePrismCompaction | experiment | Route compaction through a separate “Prism” trajectory-compaction model. | chat.conversationCompaction.prismModelFilter | model list | Which models the Prism path applies to; falls back to the agent endpoint otherwise. | Note Gone since the standalone repo: chat.backgroundCompaction and chat.advanced.agentHistorySummarizationInline . Background compaction is no longer a separate experiment — it’s on by default under the master switch. Compared to Claude Code Same problem, different taste. Claude Code’s /compact also summarizes history into a block — see The Four Types of Memory for AI Agents /posts/four-types-memory-coding-agents-claude-code/ The Four Types of Memory for AI Agents and How Claude Code Implements Each Working, semantic, procedural, episodic. The CoALA framework splits agent memory into four kinds. Here is what each one is, and how Claude Code actually implements them on disk. . Copilot leans harder on two ideas worth stealing if you build your own loop like this /posts/building-your-own-coding-agent-from-scratch/ Building Your Own Coding Agent from Scratch A practical guide to creating a minimal Claude-powered coding assistant in TypeScript. Start with a basic chat loop and progressively add tools until you have a fully functional coding agent in about 400 lines. : prune by priority before you summarize , and make the lossy summary recoverable with a breadcrumb to the raw history.