Retrieval-Augmented Generation (RAG)

How data engineers build and maintain RAG pipelines: chunking strategies, retrieval quality, and keeping knowledge bases fresh.

RAG is primarily a data pipeline problem dressed up as an AI problem. The model is not the bottleneck. The quality of the data you retrieve is.

Most failed RAG systems fail at chunking. A chunk that splits a sentence in the middle of a key term will never retrieve correctly. The right chunk size is governed by your embedding model's context window and the granularity of the questions your users ask -- not by a round number like 512 tokens.

Retrieval quality is measured, not assumed. Before you optimize anything, build an eval harness (see Evals) with 50-100 golden question-answer pairs drawn from real user queries. Run every chunking and retrieval change through it. Without evals, RAG optimization is guesswork.

Freshness is where data engineers own the problem completely. A RAG knowledge base is a derived dataset with an SLA. Define it: how stale is too stale? Build the ingestion pipeline that enforces that SLA -- incremental updates, document-level invalidation, version tracking. Treat it like any other data product.

Hybrid retrieval (dense + sparse) consistently outperforms pure vector search on diverse document corpora. BM25 catches exact keyword matches that embedding similarity misses. Most production RAG systems that have been optimized use both.

The hardest RAG problems are organizational: who owns the source documents? Who validates that the knowledge base is accurate? Who gets paged when retrieval quality drops? Solve the data ownership problem before you solve the embedding problem.