.NET Compaction - Introducing compaction strategies and pipeline

[crickman]

Motivation and Context

Context management (compaction) is one of the key features in the "dev-harness" effort. This change introduces structured handling of long-running AI chat conversations by compacting historical context while preserving key decisions and intent. By reducing token growth and context drift, it improves response quality, performance, and cost predictability over extended sessions. The implementation is designed to be extensible and transparent, making context lifecycle management a first‑class concern for agent development.

[Spec]

Description

The goal of this approach is inject compaction as part of the IChatClient architecture. This approach maintains an index that allows for presenting a compacted version of the conversation without modifying the source "chat history".

Features

• Built-in compaction strategies
• Supports pipeline strategy
• Supports direct invocation
• Supports IChatReducer
• State retained with session serialization
• Incremental message processing
• Atomic tool-call grouping
• Includes tool loops
• Retains original chat history

Details

Compaction occurs via a CompactionStrategy. A set of strategies are incuded as part of this initial release, including a pipeline strategy that is able to sequentially apply one or more strategies:

Strategy	Aggressiveness	Preserves context	Requires LLM	Best for
ToolResultCompactionStrategy	Low	High — only collapses tool results	No	Reclaiming space from verbose tool output
SummarizationCompactionStrategy	Medium	Medium — replaces history with a summary	Yes	Long conversations where context matters
SlidingWindowCompactionStrategy	High	Low — drops entire turns	No	Hard turn-count limits
TruncationCompactionStrategy	High	Low — drops oldest groups	No	Emergency token-budget backstops
PipelineCompactionStrategy	Configurable	Depends on child strategies	Depends	Layered compaction with multiple fallbacks
ChatReducerCompactionStrategy	Configurable	Depends on the IChatReducer	Depends	Compact using an existing IChatReducer

Code

// Setup a chat client for summarization
IChatClient summarizingChatClient = ...;

// Configure the compaction pipeline with one of each strategy, ordered least to most aggressive.
PipelineCompactionStrategy compactionPipeline =
    new(// 1. Gentle: collapse old tool-call groups into short summaries like "[Tool calls: LookupPrice]"
        new ToolResultCompactionStrategy(CompactionTriggers.TokensExceed(0x200)),

        // 2. Moderate: use an LLM to summarize older conversation spans into a concise message
        new SummarizationCompactionStrategy(summarizingChatClient, CompactionTriggers.TokensExceed(0x8000)),

        // 3. Aggressive: keep only the last N user turns and their responses
        new SlidingWindowCompactionStrategy(CompactionTriggers.TurnsExceed(4)),

        // 4. Emergency: drop oldest groups until under the token budget
        new TruncationCompactionStrategy(CompactionTriggers.GroupsExceed(0x1000)));

AIAgent agent =
    agentChatClient.AsAIAgent(
        new ChatClientAgentOptions
        {
            Name = "ShoppingAssistant",
            ChatOptions = new()
            {
                Instructions = "...",
                Tools = [...],
            },
            AIContextProviders = [new CompactionProvider(compactionPipeline)],
        });

or

AIAgent agent =
    agentChatClient
        .AsBuilder()
        .UseAIContextProviders(new CompactionProvider(compactionPipeline))
        .BuildAIAgent(
            new ChatClientAgentOptions
            {
                Name = "ShoppingAssistant",
                ChatOptions = new()
                {
                    Instructions = "...",
                    Tools = [...],
                }
            });

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 4 comments.

[westey-m]

Consider also adding the impact of this choice here, similar to in the sample itself, i.e. that it doesn't do in function loop compaction.

[alliscode]

Agreed, we need to find a way to make the decision on which compaction system (MEAI vs this) to use very clear in samples and docs.

[westey-m]

I've read the comments for SlidingWindow and Truncation, and it's not clear to me why I would pick the one over the other. In terms of naming I would have thought that truncation is a type of sliding window as well, in that it's essentially a window of the most recent x messages. Is there some additional core functionality that we should highlight that differentiates the two, both in naming and xml docs?

[crickman]

Yes, agree they are both similiar. I struggled with that as well. As it stands, truncation is more granular in that it operates at the message (message group) level, while sliding window lops of an entire user turn. (SlidingWindow more agressive than Truncation)

I'll re-review the original requirements/analysis and also see if comments can't be improved.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 4 comments.

[Copilot]

toolNames may include duplicates (e.g., repeated tool calls in the same assistant message or multiple function-call contents). Consider de-duplicating (and possibly ordering) the tool names before generating the summary string so the collapsed output is stable and easier to read/debug.

[Copilot]

toolNames may include duplicates (e.g., repeated tool calls in the same assistant message or multiple function-call contents). Consider de-duplicating (and possibly ordering) the tool names before generating the summary string so the collapsed output is stable and easier to read/debug.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

[alliscode]

Is there a way to cache any of these computations such that we don't iterate the entire index for every check?

[crickman]

Good call out...the thought had crossed my mind, but I was hesitant to get too fancy. I think we can definately do an efficiency improvement here...i've already got an idea, but don't want to slip Atul's ship goal:

#4605

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.

[Copilot]

The README says the sample prints the in-memory message count so you can observe the pipeline compacting the history, but the code prints the InMemoryChatHistoryProvider stored history count (which generally continues to grow) rather than the compacted message set sent to the model. Either adjust the text to clarify it’s printing stored history growth, or change the sample to report the compacted message count/view.

Suggested change

	The sample runs a seven-turn shopping-assistant conversation with tool calls. After each turn it prints the current in-memory message count so you can observe the pipeline compacting the history as the conversation grows.
	The sample runs a seven-turn shopping-assistant conversation with tool calls. After each turn it prints the total number of messages stored in the in-memory history provider. This stored history count will generally grow over time, while the compaction pipeline independently controls how many of those messages are actually sent to the model in each turn.

[Copilot]

SummarizationCompactionStrategy inserts the summary group without a turnIndex, so it ends up with TurnIndex = null. That means it won’t participate in turn-based metrics/strategies (e.g., sliding-window compaction) and may be preserved differently than intended. Consider passing an appropriate turn index (for example, the TurnIndex of the first summarized group) when inserting the summary group.

Suggested change

	index.InsertGroup(insertIndex, CompactionGroupKind.Summary, [summaryMessage]);
	// Use the turn index of the first summarized group so the summary participates in turn-based strategies
	var summaryTurnIndex = excludedGroups.Count > 0 ? excludedGroups[0].TurnIndex : null;
	index.InsertGroup(insertIndex, CompactionGroupKind.Summary, [summaryMessage], summaryTurnIndex);

[Copilot]

The sample output label prints "[Messages: x{history.Count}]"; the x looks accidental and makes the count harder to read. Consider changing it to just "[Messages: {history.Count}]" (or similar).

Suggested change

	Console.WriteLine($"\n[Messages: x{history.Count}]\n");
	Console.WriteLine($"\n[Messages: {history.Count}]\n");