Context Engineering: How to Give an AI the Right Information at the Right Moment

Many AI failures blamed on the model are really failures of information supply. The system receives the wrong policy, too many retrieved documents, stale account details, or instructions buried beneath irrelevant text. It then produces an answer that is fluent but operationally wrong.

Context engineering addresses this layer directly. It is the practice of deciding what information a model should receive, in what form, in what order, and at what stage of a task. Prompt writing remains part of the work, but the larger objective is to construct a reliable working environment around each model call.

The governing principle is simple: the best context is not the largest available context. It is the smallest context that preserves everything needed for the next decision.

What belongs inside a model’s context

A production AI system rarely works from a single user message. It may combine persistent instructions, live application state, retrieved knowledge, tool results, conversation history, and an output contract. These inputs perform different jobs and should not be treated as one undifferentiated block of text.

This separation matters because each layer has a different source of truth and refresh cycle. Rules may change monthly. Account state may change within seconds. Evidence may depend on the exact question. Conversation history may become less useful as a task progresses.

The four operations of context engineering

Useful context is produced through four operations: selection, compression, ordering, and isolation. Together, they turn a large information environment into a focused decision surface.

Selection

Selection determines which information enters the context at all. A retrieval system should not merely find text with similar vocabulary. It should find evidence capable of changing the answer. For a refund request, the purchase date, product category, payment status, and applicable policy may matter. The customer’s marketing preferences probably do not.

Compression

Compression reduces information while preserving decision-relevant facts. This can mean converting a long account record into explicit fields, summarizing earlier dialogue, or extracting policy clauses from a document. Good compression retains provenance: the system should still know where a condensed fact came from.

Ordering

Ordering makes the hierarchy legible. Stable rules should be clearly separated from untrusted source text. The immediate task should be easy to locate. Evidence should sit near the question it supports. When information conflicts, precedence should be explicit rather than left for the model to infer.

Isolation

Isolation prevents one kind of content from impersonating another. A retrieved webpage may contain instructions, but those instructions are evidence to inspect, not commands to obey. Tool output may contain user-authored text. Treating all inserted text as equally authoritative creates both reliability and security problems.

A worked example: resolving a damaged-order request

Consider a support assistant handling this message: “The ceramic lamp arrived cracked. Can you send another one before Friday?”

A weak implementation might attach the entire conversation, the full customer record, several shipping pages, and the complete returns manual. A better implementation first identifies the decisions required:

1. Is the damage claim eligible for replacement?
2. Is the item currently available?
3. Can a replacement arrive before the requested date?
4. If not, which alternatives may be offered?

The system then gathers only the facts needed for those decisions:

• Order state: delivered ceramic lamp, damage reported after delivery.
• Eligibility rule: the relevant damaged-goods clause, including any evidence requirement.
• Inventory state: stock for the exact item and permissible substitutes.
• Fulfilment result: estimated arrival dates for the customer’s destination.
• Authority boundary: whether the assistant may create a replacement or must request approval.

The assembled context might use a compact structure:

Task: Determine the next permitted action for a damaged delivered item.
Rules: Apply the damaged-goods policy. Never promise an arrival date that has not been returned by the fulfilment tool. Ask for missing mandatory evidence.
Order state: Item matches order. Delivery confirmed. No prior replacement.
Policy evidence: Replacement is permitted after required damage evidence is recorded.
Inventory: Exact item available.
Fulfilment: Earliest estimated arrival is after Friday.
Required output: eligibility, action, customer message, unresolved requirement.

The assistant can now give a precise response: the item is eligible in principle, the required damage evidence must be collected if absent, and Friday delivery cannot be promised. The crucial improvement does not come from rhetorical prompt refinement. It comes from aligning the context with the decisions.

Build context around decisions, not documents

A common architectural mistake is to treat documents as the natural unit of context. Models are then given whole policies, long tickets, or complete database records. Yet the task usually depends on a handful of claims rather than an entire source.

Instead, define a decision schema before designing retrieval. For the replacement example, the schema could contain:

• claim eligibility status;
• missing evidence;
• replacement availability;
• validated delivery estimate;
• permitted next action;
• supporting source references.

This schema reveals what must be fetched and what can be excluded. It also exposes missing information. If the fulfilment tool returns no estimate, the system should mark delivery timing as unresolved rather than inviting the model to infer it.

Structured state is particularly valuable for frequently changing facts. Instead of embedding a paragraph saying an item “appears to be available,” pass an explicit inventory status and timestamp from the responsible system. Narrative text remains useful for nuanced policy, but operational state should usually arrive as data.

Control context growth across multiple steps

Agentic workflows create a second problem: context accumulates. Tool calls, intermediate reasoning, retrieved pages, and partial drafts are repeatedly appended until the model must navigate its own debris.

Use a context budget for each step. The budget need not begin as a token number. It can be an information policy:

• Carry forward confirmed facts, unresolved questions, commitments, and source references.
• Discard duplicate retrieval results, superseded plans, and verbose tool narration.
• Replace completed exchanges with a state summary.
• Fetch detailed evidence again when needed rather than preserving everything indefinitely.

This creates a distinction between working context and durable state. Working context is temporary material needed for the next model call. Durable state belongs in databases, workflow engines, or event logs. The model should receive a relevant projection of that state, not become its permanent storage layer.

Test the context, not only the final answer

Answer-level evaluation tells you whether a system failed. Context-level evaluation helps explain why. Record the assembled context for each run and inspect it as a first-class artefact.

Useful tests include:

• Relevance: Did every included item support a decision?
• Coverage: Were all facts required for the correct action present?
• Freshness: Did mutable facts come from current systems?
• Authority: Were rules, evidence, and untrusted content clearly distinguished?
• Conflict handling: Was precedence explicit when sources disagreed?
• Compression fidelity: Did summaries preserve exceptions, dates, and constraints?

Counterfactual tests are especially revealing. Remove one context item and run the task again. If nothing changes, the item may be unnecessary. Replace a decisive fact, such as inventory status, and verify that the action changes appropriately. Insert an irrelevant instruction inside a retrieved document and confirm that the system treats it as content rather than authority.

A practical implementation sequence

Context engineering is easiest to introduce around one bounded workflow rather than as an abstract platform initiative.

1. Name the decision. Write the exact judgment or action the model must produce.
2. List required facts. Identify what a careful human would need, including exceptions and permissions.
3. Assign sources of truth. Map each fact to a database, tool, policy repository, user input, or conversation state.
4. Define precedence. Decide what wins when cached state, user claims, and live systems conflict.
5. Choose representations. Use structured fields for state, excerpts for evidence, and summaries for history.
6. Assemble in layers. Keep rules, task, state, evidence, and output requirements visibly separate.
7. Instrument the result. Store context manifests, source identifiers, timestamps, and model outputs for evaluation.
8. Prune deliberately. Remove material that does not improve correctness, safety, or auditability.

The deeper opportunity is architectural. Once context is treated as a designed product rather than a prompt attachment, model upgrades become easier to evaluate, retrieval becomes accountable, and agent behavior becomes less mysterious. The model still supplies intelligence, but the system determines what that intelligence is allowed to see—and therefore what it can reliably decide.

This post was drafted with AI assistance and reviewed against our editorial policy before publication. Corrections are made at the source, on the page, with the date shown.