How to reliably extract structured data from LLMs using schema constraints, JSON mode, and function calling -- the data engineer's guide to type-safe AI outputs.
Structured output extraction is the most common data engineering task in AI systems, and the most frequently underengineered. Asking a model to "return JSON" in a system prompt works most of the time -- until it does not, and your downstream pipeline receives a markdown code block, a preamble paragraph, or a truncated object. Schema-constrained generation eliminates this failure class by restricting the model's token choices at inference time to only those that produce valid output for your schema.
The two architectural approaches are constraint-at-inference (restricting valid tokens via guided decoding) and constraint-at-validation (letting the model output freely, then parsing and retrying on failure). Managed APIs like OpenAI Structured Outputs and Anthropic tool use implement constraint-at-inference internally. Libraries like Instructor add constraint-at-validation with retry logic on top of any API. Use whichever your provider supports natively; the retry-based approach adds latency and cost on every failure.
Schema design is a prompt engineering problem that most teams underinvest in. A schema that is too broad -- every field optional, permissive types -- does not help the model understand what you need. A schema that is too strict fails on legitimate variation. The sweet spot is schemas that match the semantic structure of the information you are extracting, with required fields only for information that is always present. Description fields in JSON Schema are not decoration -- models read them, and clear descriptions measurably improve extraction accuracy.
Nested extraction is where structured outputs provide the most leverage over ad-hoc parsing. Extracting entities with their attributes from unstructured text into a typed schema replaces hundreds of lines of regex and brittle string manipulation. The result is a reliable ETL step with a versioned schema, testable behavior, and clear ownership. Treat your extraction schema as a data contract with all the engineering discipline that implies.
The clearest production-ready implementation reference. Covers JSON Schema constraints, supported schema features, and refusal handling. Read this before building any extraction pipeline on the OpenAI API.
Model-agnostic constrained generation library for local and open-weight models. The mechanism -- logit biasing via finite-state machines compiled from your schema -- is worth understanding even if you use a managed API. Self-hosted extraction pipelines rely on this.
Python library that wraps OpenAI, Anthropic, and other APIs with Pydantic validation and automatic retry on parse failure. The fastest path to type-safe structured extraction in production Python.
Anthropic's function-calling interface for structured extraction. The tool_use content block delivers reliable JSON without requiring the model to emit valid JSON inside its text response -- avoids the markdown-wrapper failure mode.
New resources and perspective on building AI-ready data systems, a few times a month. No spam.