BigQuery Data Lineage: Trace Columns Across Your Datasets

BigQuery gives you three native ways to trace data lineage: the INFORMATION_SCHEMA views for object metadata, Cloud Audit Logs and INFORMATION_SCHEMA.JOBS for query history, and the built-in Data Lineage API for an automatically generated dependency graph that has resolved columns, not just tables, since September 2025. The native graph is better than its reputation. Where it runs out is durability and reach: lineage is retained for only 30 days, it stops collecting column detail on any job that creates more than 1,500 column links, it can take 24 hours to appear, and it ends at the edge of Google Cloud.

This guide walks through what BigQuery offers natively as of July 2026, the documented limits of each approach, and how to get column-level lineage that persists and follows your data into the dashboards downstream. If you want the product view rather than the how-to, see our BigQuery data lineage page.

Starting with INFORMATION_SCHEMA

The most direct place to begin is the INFORMATION_SCHEMA views, scoped per dataset or per region. INFORMATION_SCHEMA.TABLES and INFORMATION_SCHEMA.COLUMNS give you the schema of every table and view: column names, types, and nullability. For views, INFORMATION_SCHEMA.VIEWS exposes the view definition SQL, and for routines there is INFORMATION_SCHEMA.ROUTINES.

This is enough to map static structure. If analytics.fct_orders is a view, you can read its definition and see it selects from analytics.stg_stripe_charges. The limitation is immediate: INFORMATION_SCHEMA tells you about objects as they exist now, but it does not tell you how a materialized table was populated. A scheduled query or dbt model that runs CREATE OR REPLACE TABLE fct_orders AS SELECT ... leaves no view definition behind. The table is just a table. To recover that lineage you have to look at the queries that built it.

Reconstructing lineage from query history

The richest native source of lineage is query history, available two ways. Cloud Audit Logs capture every BigQuery job, including the SQL text and the referenced and destination tables. INFORMATION_SCHEMA.JOBS (and JOBS_BY_PROJECT) exposes the same job metadata in queryable form, including referenced_tables and the destination table for each job.

By parsing this history you can reconstruct table-level lineage: every job that wrote to fct_orders read from stg_stripe_charges and dim_customers, so those are its parents. This is powerful because it captures real data flow, not just declared structure, and it picks up the scheduled queries and ad hoc loads that INFORMATION_SCHEMA misses.

The work is non-trivial, though. You need to:

• Continuously export and retain job history, because INFORMATION_SCHEMA.JOBS only retains a window (typically 180 days).
• Parse SQL to go beyond referenced_tables into column-level relationships, which the audit log does not break out for you.
• Deduplicate and reconcile many jobs writing to the same destination over time.

The result is table-level lineage that you maintain yourself. Getting from there to "which source column feeds orders.revenue" means writing a SQL parser that understands joins, aliases, CTEs, and SELECT * expansion across every dialect quirk in your codebase.

The native Data Lineage API (now Knowledge Catalog)

Google's managed answer is the built-in data lineage feature, surfaced through the Data Lineage API and visible in the BigQuery console. The naming here has moved fast, and most articles you will find are out of date: Data Catalog was deprecated in February 2025 and scheduled for shutdown on 1 June 2026, its catalog role passed to Dataplex, and as of 10 April 2026 Dataplex Universal Catalog is called Knowledge Catalog. The API and IAM names did not change, so you still enable datalineage.googleapis.com and grant roles/datalineage.viewer regardless of what the console calls it this quarter.

Once the API is enabled on a project, lineage is captured automatically for query jobs, copy jobs, and load jobs from Cloud Storage, plus Dataflow, Spark, Composer, and Looker. No parser to write.

And it is column-level. This is the fact most competing articles get wrong, so it is worth stating plainly: column-level lineage for BigQuery went generally available on 29 September 2025, covering CREATE TABLE, INSERT, UPDATE, MERGE, DELETE, and SELECT with a destination table. If you read somewhere that BigQuery only does table-level lineage, that page is stale.

The real limits are more specific, and more interesting:

• Lineage expires after 30 days. Google's documentation is explicit that lineage information is retained for only 30 days. A table rebuilt quarterly simply has no lineage most of the time. By contrast INFORMATION_SCHEMA.JOBS keeps job history for 180 days.
• It silently degrades past 1,500 column links. Column-level lineage is not collected if a job creates more than 1,500 column-level links. The wide fact table built by your biggest transformation is exactly the job most likely to cross that line, and you are not told when it does.
• Latency up to 24 hours. Lineage appears within 24 hours of the job completing, which rules it out as a pre-merge check.
• Nested fields are not supported. Column-level lineage covers top-level columns only. Fields inside STRUCT or JSON types are not tracked.
• Gaps in what is captured: recurring load jobs from the BigQuery Data Transfer Service are not automatically recorded, and neither is direct lineage for BigQuery routines, so your stored procedures are invisible.
• External tables do not get upstream column-level lineage.
• It is not free. Lineage bills against the Dataplex premium processing SKU at $0.089 per DCU-hour plus $2 per GiB of metadata storage, and the free tier explicitly does not apply to the premium SKU.
• It stops at the edge of Google Cloud. Transformations outside BigQuery need custom lineage events reported through the API to appear at all.

For a governance overview inside the Google console, native lineage is genuinely useful, and if your whole estate is BigQuery it may be all you need. The friction shows up when the graph you needed aged out three weeks ago, quietly downgraded on your largest job, or stopped at the boundary of the dashboard someone actually opens.

Where the native tools collectively stop

Put together, BigQuery's native lineage story has three recurring gaps:

1. Durability and completeness: INFORMATION_SCHEMA and audit logs resolve to tables, not columns, so getting from fct_orders depends on stg_stripe_charges to orders.revenue derives from stg_stripe_charges.amount means writing a parser. The native API does resolve columns, but only for 30 days and only below the 1,500-link ceiling.
2. Staying current: lineage you reconstruct by hand is a snapshot. The moment someone refactors a model or renames a column, your graph is stale until you rerun your pipeline.
3. Crossing the boundary: real impact lives downstream in dashboards, exposures, and reverse-ETL syncs. Native BigQuery lineage rarely reaches those consumers.

Automatic column-level lineage that stays current

This is the gap Datatrail is built to close for BigQuery. It connects to your project read-only, never moving or mutating data, and continuously parses your query history to build column-level lineage automatically. Rather than a 180-day window you have to export, it maintains a graph that refreshes as new jobs run, so the lineage you see matches the warehouse as it is today, not as it was last quarter.

Because the graph is column-level, you can trace orders.revenue back through fct_orders to the exact source field, and then look the other direction with impact analysis to see every downstream model and dashboard before you change it. Layered on top are freshness monitoring for tables that quietly stop loading and schema change alerts for the renamed or dropped columns that BigQuery will happily let break a dashboard overnight.

If you maintain a BigQuery warehouse and have hit the wall where INFORMATION_SCHEMA.JOBS and Dataplex get you most of the way but not to the column, you can see how Datatrail builds lineage from your query logs and map your own datasets on a live platform.