Context Engineering

Context engineering is the discipline of designing what information goes into an LLM's context window -- and how. A data engineer's guide.

Context engineering is the most direct lever data engineers have on AI system quality. The model is fixed; the context is yours to design. What you put in the context window, in what order, in what format, determines most of what you get out.

The fundamental tension in context engineering is relevance vs. completeness. Stuffing the context with everything possibly related improves completeness but degrades relevance -- the model spreads attention across irrelevant material. Relevance almost always wins. Ruthlessly cut.

Ordering matters more than most engineers expect. LLMs attend more strongly to the beginning and end of their context (the "lost in the middle" problem). Put the most decision-critical information at the top. Put the user's question immediately before the generation step. Test this empirically with your eval harness.

Structured formats (JSON, Markdown tables, numbered lists) improve reliability for structured extraction tasks. Prose is better for open-ended generation. Match the context format to the output you need. Consistent delimiters (XML tags, triple-backtick code blocks) help models reliably parse multi-section prompts.

Context engineering and RAG are inseparable. What you retrieve is context engineering. How you format the retrieved chunks is context engineering. Whether you include source metadata is context engineering. Every decision about what to put in the prompt is a context engineering decision.

The state of the art is dynamic context: assembling the context at request time from multiple sources -- user history, retrieved documents, tool outputs, structured data from your data warehouse -- rather than a static prompt template. Data engineers build the pipelines that populate that context. This is the data engineer's emerging role in AI product development.