Context compression

Compression shrinks context with a specialised LLM before the main model reads it. Overflow managementContext overflowWhen the history outgrows the window: cut the oldest, or summarize them.Opens in a new tab decides that the history is too long; compression is the how: a model that does nothing but summarize.

In LangChain

SummarizationMiddleware runs before the model call. When the history reaches the trigger, everything older than the messages to keep is replaced by a summary. The summarizer is a smaller, cheaper model, so the big model only pays for the shorter history.

from langchain.agents import create_agentfrom langchain.agents.middleware import SummarizationMiddlewarefrom langgraph.checkpoint.memory import InMemorySaver agent = create_agent(    model="anthropic:claude-sonnet-5-5",    tools=[],    middleware=[        SummarizationMiddleware(            model="anthropic:claude-haiku-4-5-20251001",   # a smaller model does the summarizing            trigger=("tokens", 4000),                       # when the history reaches this            keep=("messages", 20),                          # the newest stay verbatim        )    ],    checkpointer=InMemorySaver(),)

Compaction

Anthropic calls the same move compaction: take a conversation nearing the limit, summarize it, and start a new window from the summary. What the summary keeps decides whether the work can continue, so it has to carry the decisions made and the problems still open, not a general recap.

The catch

Compression is lossy, and the loss is silent. What the summary drops cannot be recovered from the window. Facts the work depends on are safer as notes written outside the window and read back when needed (see agent memoryAgent memoryWhat an agent remembers: the thread it is in, and what it keeps across threads.Opens in a new tab). The same idea applies to bulky tool output or retrieved text: shrink it before it goes in, and keep the original where it can be fetched again.