How data catalogs (Dataplex, DataHub, Alation) ground AI agents in warehouse structure: semantic descriptions, lineage, ownership, and the catalog as a knowledge base.
The data catalog is the knowledge base for warehouse agents. Without it, a text-to-SQL agent knows the table names and column types (from the schema) but not what they mean, why they exist, who owns them, or how they relate to business concepts. The catalog is where that semantic layer lives. Schema RAG is retrieval over your catalog. The quality of your catalog is the ceiling on your agent quality.
Dataplex Knowledge Catalog is Google Cloud's managed approach: automatic schema discovery from BigQuery, Pub/Sub, Cloud Storage, and GCS, combined with manual business metadata enrichment. The key insight from the Dataplex documentation is that catalog metadata is a first-class data asset with its own access controls, lineage tracking, and quality metrics. Treat your catalog entries with the same rigor as your data models.
Business metadata is the hardest part and the most valuable. Technical metadata -- schema, data types, row counts, freshness timestamps -- can be harvested automatically. Business metadata -- what this table represents in the business, what the column values mean, which teams use it for which decisions, which metrics it feeds -- requires human knowledge and cannot be automated. The teams that invest in business metadata get AI-grounded agents; the teams that skip it get hallucinated schema.
Lineage is underused as an agent grounding signal. If a text-to-SQL agent knows not just the schema but the lineage -- this table is derived from these sources, transformed by this dbt model, consumed by these dashboards -- it can generate SQL that respects the data's provenance. "Show me revenue" is safer if the agent knows that the revenue table is the governed, dbt-tested version, not the raw events table that would require custom logic to interpret.
The catalog as a RAG knowledge base is a practical implementation pattern: embed table and column descriptions, index them in a vector store, retrieve the most relevant ones for each incoming question, include them in the schema context. This is schema linking operationalized. Tools like DataHub and Alation expose APIs for this; you build the retrieval pipeline on top. The catalog is the data; the agent is the consumer.
Google Cloud's managed catalog shows what a production metadata layer looks like: automatic technical metadata discovery, business metadata enrichment, fine-grained access controls, and lineage tracking. The architecture applies regardless of platform.
New resources and perspective on building AI-ready data systems, a few times a month. No spam.