Context engineering

Context engineering is deciding what goes into the context window on every model call. Prompt engineeringPrompt engineeringEngineering the instruction given to the model.Opens in a new tab writes the instruction. Context engineering curates everything the model reads alongside it: history, retrieved documents, tool definitions, state. Anthropic frames it as curating the optimal set of tokens at inference time.

The problem

The window is finite and the information is not. A bigger window is only a partial answer, for two reasons. Attention drops as the window fills: Anthropic calls it context rot, and treats attention as a budget the model spends. And models use the middle of a long input worse than its start and end (see rerankingContext rerankingModels read the middle of a long context worst, so put what matters at the edges.Opens in a new tab). Even models built for long context show it.

So the goal is a smart context assembler (see the context assemblerThe context assemblerThe step that builds the prompt from scratch on every loop.Opens in a new tab) that picks, orders and trims what goes in. Four strategies do that work, one note each:

• Overflow managementContext overflowWhen the history outgrows the window: cut the oldest, or summarize them.Opens in a new tab: the history grows too long, so cut it or summarize it.
• CompressionContext compressionShrink what goes in, using a smaller model whose only job is to summarize.Opens in a new tab: a specialised model shrinks it.
• RerankingContext rerankingModels read the middle of a long context worst, so put what matters at the edges.Opens in a new tab: what matters goes at the start or the end.
• ScopingContext scopingShow the model only what it needs for this step, tools included.Opens in a new tab: show only what is needed now.

The techniques

Technique	The idea
JIT retrieval	Keep identifiers (file paths, queries, links) and load the data with a tool when it is needed.
Windowing and pagination	Return a slice at a time, not everything.
Compaction	Summarize a conversation near the limit and start a new window from the summary.
Out-of-band context separation	Keep bulk data in state or a store, outside the prompt, and show only what the next step needs.
Sub-agents	A helper works in its own clean window and hands back a short summary.
Structuring	Label the pieces so the model can tell them apart.
Writing and retrieving for long tasks	Save notes outside the window and read them back later (see agent memory).

More context is not better context. The aim is the smallest set of tokens that makes the next step likely to succeed. For the alternative of skipping retrieval and pasting everything, see RAG versus long contextRAG versus long contextPassing the whole file avoids retrieval, until the cost, the attention and the window limit show up.Opens in a new tab.