How data engineers build production AI agent systems: the agent loop, tool use, ReAct, MCP, memory, multi-agent orchestration, and evaluation.
The agent loop is a simple idea that gets complicated fast: an LLM reasons, picks a tool, gets a result, reasons again, and repeats until it has an answer or gives up. Lilian Weng's 2023 post on LLM-powered autonomous agents remains the clearest map of the terrain -- covering the loop itself, the memory components (in-context, external, parametric), and the planning strategies that sit on top. Read it before building anything.
ReAct (Reasoning + Acting) is the pattern that made agentic LLM behavior practical. The key insight: interleave reasoning traces with tool calls so the model can observe tool outputs and adjust its plan. Most production agent frameworks implement ReAct or a close variant, even when they do not name it. The paper is short; read it directly rather than a summary.
Tool use is where data engineers own the most leverage. The model decides which tool to call; the data engineer designs what tools exist, what they do, and how their schemas are written. A well-designed tool schema -- clear parameter names, explicit descriptions, constrained types -- dramatically outperforms a vague one with the same implementation. The quality of your tool definitions is the highest-return investment in agent reliability.
The most consequential architectural decision in AI agents is whether you actually need an agent. Workflows -- fixed sequences of LLM calls and tool invocations -- are cheaper, more predictable, and easier to test than true agents. Use an agent only when the task genuinely requires dynamic routing that cannot be determined in advance. Most systems that call themselves agents should be workflows. Anthropic's "Building Effective Agents" is the clearest treatment of this distinction.
MCP (Model Context Protocol) is the emerging standard for connecting agents to tools and data sources. Think of it as the USB-C of agent connectivity: instead of a custom integration for every LLM-to-tool pair, MCP defines a single protocol that any compliant client and server can speak. Whether it becomes the dominant standard long-term is not settled, but it has enough momentum that any team building agent infrastructure should understand the protocol before evaluating MCP-compatible tooling.
Agent memory is a data engineering problem. In-context memory is fast but bounded by the context window. External memory -- vector stores, key-value stores, structured databases -- gives agents longer recall but adds latency and retrieval error. The decision of what to store, how to index it, and when to retrieve it is exactly the same class of problem as designing a data pipeline. Data engineers are better positioned to solve it than most. See the Anthropic guide on effective context engineering for agents as the practical reference.
Multi-agent orchestration multiplies both capability and complexity. Each agent hop adds latency, cost, and a new failure mode. The coordination tax is real: agents that hand off to other agents need to pass enough context to continue the task, and context compression at handoff points is where quality degrades silently. Start with a single agent. Add orchestration only when you have a specific bottleneck -- parallelism, specialization, context limits -- that a single agent cannot address. LangGraph is the most mature open-source library for building stateful multi-agent graphs.
You cannot evaluate an agent system the same way you evaluate a retrieval pipeline. Agents take variable-length paths to answers, and the intermediate steps matter as much as the final output. Build eval coverage for: final answer correctness, tool selection accuracy, unnecessary tool calls, and context drift across long trajectories. The HF Agents Course is the most complete hands-on introduction that covers building and evaluating agents end to end. Cross-link your agent evals to the Evals branch -- the infrastructure is the same.
The definitive map of the agent loop: components, memory types, planning strategies. Required reading before building anything.
The paper that established interleaved reasoning + tool use as the dominant agent pattern. Short and worth reading directly.
The practical guide to tool schema design. The quality of your tool descriptions matters more than you expect.
The clearest treatment of when to use workflows vs. true agents. Read before committing to an agentic architecture.
The emerging standard for connecting agents to tools and data sources. Understand the protocol before evaluating MCP-compatible tooling.
How context design affects agent reliability: what to include, what to prune, and how to structure multi-turn agent memory.
A concrete multi-agent architecture from Anthropic's engineering team, covering orchestration patterns and evaluation.
The most mature open-source library for building stateful multi-agent graphs. Start here for orchestration implementations.
The most complete hands-on introduction to building and evaluating agents. Covers the full stack from loop mechanics to deployment.
New resources and perspective on building AI-ready data systems, a few times a month. No spam.