Context Window Management for Long-Running Agents: Compaction Strategies Compared

Every long-running agent eventually hits the same wall: the conversation grows past what fits in the context window. What happens next is a design decision with real tradeoffs — not a solved problem your framework quietly handles correctly by default.

Strategy 1: Sliding window (drop the oldest turns)

The simplest approach — once you’re near the limit, drop turns from the start of the conversation until you’re back under budget.

def sliding_window(messages, max_tokens, count_tokens):
    while count_tokens(messages) > max_tokens and len(messages) > 1:
        messages.pop(0)
    return messages

Cost: whatever was in the dropped turns is gone, completely, with no chance to recover it later. This is fine for genuinely stateless interactions and actively dangerous for anything where an early decision matters forty turns later — which is exactly the failure mode in why your agent forgets things mid-task.

Strategy 2: Summarize-and-replace

Instead of dropping old turns outright, replace a block of them with an LLM-generated summary, keeping the gist at a fraction of the token cost.

def summarize_and_replace(messages, cutoff_index, llm_call):
    to_summarize = messages[:cutoff_index]
    summary = llm_call(f"Summarize this conversation concisely:\n{to_summarize}")
    return [{"role": "system", "content": f"Earlier context: {summary}"}] + messages[cutoff_index:]

Cost: the summarizer decides what’s important, and it’s a separate model call with its own failure mode — it can drop a specific detail (an exact file path, an exact number) that the narrative doesn’t flag as significant but that a later tool call depends on. This is the exact problem pre-compaction checkpointing (see agent-memory-kit’s CompactionCheckpoint, linked once the repo is public) exists to guard against: extract structured facts before the lossy summarization step runs, not after.

Strategy 3: Structured extraction into persistent state

Rather than summarizing the narrative, extract specific structured facts — decisions, preferences, open tasks — into state that lives outside the conversation entirely, then let the raw turns be dropped or summarized freely because the load-bearing facts already survived elsewhere.

extracted = llm_call("Extract only: decisions made, preferences stated, open tasks. JSON only.", to_summarize)
agent_state.update(extracted)  # ScratchpadState from agent-memory-kit
messages = sliding_window(messages, max_tokens, count_tokens)  # now safe to drop freely

Cost: requires deciding in advance what counts as “load-bearing,” which is a real design decision, not automatic. Undercover this scope and you’ve reinvented summarize-and-replace with extra steps; overscope it and the state block grows unbounded and stops being cheap.

Strategy 4: Hierarchical/tiered context (recent verbatim, older compressed progressively)

The most sophisticated of the four: keep the most recent N turns completely untouched, summarize the next tier back, and drop or heavily compress anything older than that — so recency gets full fidelity and only distant history pays the compression cost.

def tiered_context(messages, recent_n=10, mid_tier_n=30):
    recent = messages[-recent_n:]
    mid = messages[-mid_tier_n:-recent_n]
    old = messages[:-mid_tier_n]
    return [summarize(old), *[light_summarize(m) for m in mid], *recent]

Cost: the most implementation complexity of the four, and it still inherits summarize-and-replace’s core risk for anything pushed into the mid or old tiers — it just delays when that risk applies rather than removing it.

Picking one

In practice, strategy 3 (structured extraction) combined with a plain sliding window for the raw conversation is the right default for most agents — it isolates the one thing that actually needs to survive compaction (specific facts) from the one thing that’s safe to lose (the exact wording of how you got there), without the added complexity of full tiering. Reach for tiering only once you’ve confirmed recency fidelity in the mid-range is an actual, measured problem — not before.