Context overflow

Overflow management deals with the one kind of context that grows by itself: the episodic history. Left alone, a long conversation outgrows the window, so something has to give. There are two moves.

Sliding window

Keep the newest messages and drop the rest. It is cheap and predictable, and whatever falls out is gone: the user’s name from message two, a constraint stated early. LangChain ships trim_messages for it.

from langchain_core.messages import trim_messages def window(messages):    return trim_messages(        messages,        strategy="last",          # keep the newest        token_counter=len,        # count messages instead of tokens        max_tokens=10,        start_on="human",         # never begin mid-exchange        include_system=True,      # keep the system message    )

strategy="last" keeps the newest. start_on and include_system keep the window from opening on a reply whose question was cut, and keep the instructions. Here len counts messages; a real token count can come from a model or from "approximate".

Summarization

Instead of dropping the old messages, replace them with a summary of them. The model keeps the gist at the price of one more model call, and detail the summary leaves out is lost just the same. How that works is in compressionContext compressionShrink what goes in, using a smaller model whose only job is to summarize.Opens in a new tab.

Choosing

Use a window when old turns rarely matter. Summarize when they carry decisions. Neither is a memory: anything that must survive belongs in a store, not in a message that may be trimmed (see agent memoryAgent memoryWhat an agent remembers: the thread it is in, and what it keeps across threads.Opens in a new tab).