Datatrail

Blog

Data lineage and data observability, explained

Guides and practical playbooks for mapping data flow table-to-table, tracing columns to their source, monitoring freshness, catching schema changes, and seeing downstream impact before a change breaks a dashboard.

Guides 8 min read

Data Observability vs Monitoring: The Difference and When You Need Each

Data monitoring checks conditions you defined in advance; data observability learns each table normal behavior and flags the failures you never wrote a rule for. The real difference, why it matters at buying time, and the lineage layer that turns either one into action.

Read article
Guides 8 min read

Great Expectations vs dbt Tests: Which Data Quality Approach Fits

dbt tests are SQL assertions inside your dbt project; Great Expectations is a standalone Python framework that validates data anywhere, including before it lands. When to use each, how to split them by pipeline stage, and the lineage layer neither one gives you.

Read article
Guides 9 min read

Looker Data Lineage: The LookML Graph, Knowledge Catalog, and Its Limits

How data lineage works in Looker, the PDT dependency graph and the object-level Knowledge Catalog integration, why neither is column-level, and how to trace a LookML measure to the warehouse column behind it.

Read article
Guides 8 min read

Collibra Pricing: How Much Does Collibra Cost in 2026?

Collibra is quote-only. What independent procurement data says a typical deployment costs, what drives the price, and when a lighter, transparently priced lineage tool is the better fit.

Read article
Guides 8 min read

Snowflake Data Quality Checks: DMFs, dbt Tests, and What Each One Misses

How to run data quality checks in Snowflake with data metric functions, the documented limits of DMFs, where dbt tests stop, and why quality alerts need column-level lineage to be actionable.

Read article
Guides 9 min read

Automated Data Quality Monitoring: How It Works and Where Lineage Fits

How automated data quality monitoring learns normal behavior and flags anomalies without per-table rules, the three layers that make it work, and why it needs lineage to be actionable.

Read article
Guides 9 min read

Power BI Data Lineage: The Lineage View, Its Limits, and Column-Level Tracing

How to see data lineage in Power BI with the workspace Lineage view and impact analysis, why it stops at the semantic model, and how to get column-level lineage from the warehouse behind it.

Read article
Guides 9 min read

Tableau Data Lineage: Tableau Catalog, Field-Level Lineage, and Its Boundaries

How Tableau Catalog builds column-level data lineage, why it needs the Data Management add-on, where custom SQL breaks it, and how to trace the warehouse and dbt path upstream of Tableau.

Read article
Guides 11 min read

Data Lineage for BI Tools: Power BI, Tableau, and Looker Compared

The built-in data lineage in Power BI, Tableau, and Looker each stops at its own boundary. Why warehouse-native column lineage underneath all three gives you one graph and real impact analysis.

Read article
Guides 10 min read

How to Handle Schema Drift: Detect It, Rank It, and Fix It Before It Breaks Reports

A practical playbook for handling schema drift: how to detect structural changes from warehouse metadata, why raw alerts get muted, and how to rank drift by downstream impact so you fix the changes that actually matter.

Read article
Guides 11 min read

Airflow Data Lineage: OpenLineage, Marquez, and What They Do Not Capture

How to get data lineage out of Apache Airflow with the native OpenLineage provider and Marquez, what column-level lineage you actually get, and the coverage gaps that warehouse-native lineage fills.

Read article
Guides 10 min read

How to Visualize dbt Lineage: The DAG, Its Limits, and Column-Level Graphs

How to visualize the data lineage dbt creates, from the built-in docs DAG to column-level graphs, why the dbt DAG stops at your project boundary, and how to see the full picture out to the dashboards.

Read article
Guides 11 min read

Schema Change Impact Analysis: How to See What Breaks Before You Deploy

A practical method for finding the blast radius of a schema change before it ships: map the dependents, separate technical from business impact, and coordinate the deploy so nothing downstream breaks by surprise.

Read article
Fundamentals 10 min read

Data Lineage vs Data Catalog: What Is the Difference and Which Do You Need?

A data catalog tells you what data exists and a data lineage tool tells you how it connects and what breaks when it changes. Here is how they differ, where they overlap, and how to decide which one your team needs first.

Read article
Guides 10 min read

Redshift Data Lineage: How to Build It From Query History

How to trace column-level lineage in Amazon Redshift from the system tables, why the seven-day log retention is the catch, and how AWS-native and dedicated tools each fill the gap.

Read article
Guides 11 min read

Schema Drift Detection: Catch a Breaking Column Change Before It Ships

How to detect schema drift with the metadata your warehouse already exposes, why the naive snapshot-and-diff version gets muted within a week, and how lineage turns drift into an alert worth reading.

Read article
Guides 10 min read

dbt Dependency Graph: How to See It, Use It, and Where It Stops

How dbt builds its DAG from ref and source calls, the three ways to read it, the graph operators worth knowing, and the four things that live outside the graph entirely.

Read article
Guides 11 min read

Root Cause Analysis for Data Pipelines: A Four-Step Method

A repeatable method for tracing a wrong number back to the change that caused it: pin the symptom, walk the lineage, check the three failure classes, and confirm with timestamps.

Read article
Fundamentals 10 min read

What Is Data Lineage? A Plain-English Guide for Data Teams

What data lineage is, why it matters, and how table and column-level lineage let you trace any number to its source and see what breaks before a change ships.

Read article
Fundamentals 10 min read

What Is Data Observability? The Five Pillars Explained

Data observability is how you know your data is fresh, complete, and correct. Learn the five pillars, how it differs from monitoring, and where lineage fits.

Read article
Guides 9 min read

Data Lineage Example: Tracing Revenue From Source to Dashboard

A concrete data lineage example: follow orders.revenue from a raw Stripe table through staging and models to the dashboards that depend on it, column by column.

Read article
Guides 8 min read

Data Lineage Diagram: How to Read and Build One

A data lineage diagram maps how data flows from source to dashboard. Learn how to read nodes and edges, what to include, and why hand-drawn diagrams go stale.

Read article
Reliability 8 min read

What Is Data Downtime, and What Does It Cost You?

Data downtime is any period when your data is wrong, missing, or stale. Learn how it happens, how to measure it, and how to catch it before a person does.

Read article
Playbooks 11 min read

How to Build Data Lineage: Manual, dbt, and Automated Approaches

How to build data lineage three ways: by hand, from dbt, and automatically from query logs. The tradeoffs, the staleness problem, and what column-level adds.

Read article
Fundamentals 9 min read

Data Observability vs Data Quality: What Is the Difference?

Data observability and data quality are related but not the same. Learn how monitoring, testing, and lineage fit together, and where each one catches problems.

Read article
Buyer Guides 11 min read

Best Data Observability Tools in 2026: A Buyer's Guide

The best data observability tools compared on lineage depth, freshness and schema monitoring, impact analysis, pricing model, and time to value. How to choose.

Read article
Guides 9 min read

BigQuery Data Lineage: Trace Columns Across Your Datasets

How BigQuery data lineage works, what INFORMATION_SCHEMA and audit logs give you, where they stop, and how to get automatic column-level lineage that stays current.

Read article
Fundamentals 9 min read

Column-Level Lineage Explained: Why Table-Level Is Not Enough

Column-level lineage traces a single field through every transformation. Learn why table-level lineage misses breaks, and how column lineage answers what feeds a number.

Read article
Guides 10 min read

Snowflake Data Lineage: A Practical Guide to Tracing Data Flow

How Snowflake data lineage works with ACCESS_HISTORY and QUERY_HISTORY, what you can build yourself, and how to get automatic column-level lineage and freshness alerts.

Read article
Guides 9 min read

Downstream Impact Analysis of Data Changes: Find Everything a Column Change Will Break

How to find every downstream dependency of a table before you change it: query history, column-level parsing, and following the chain out to the dashboards people actually open.

Read article
Platforms 9 min read

Unity Catalog Lineage: What Databricks Data Lineage Captures and What It Misses

Unity Catalog captures column-level lineage on Databricks for free. Here is exactly what it covers, the documented gaps (renames, UDFs, path reads, the one-year window), and how to fill them.

Read article
Buying 8 min read

Data Observability Pricing: What It Costs and What Actually Drives the Quote

Most data observability vendors do not publish pricing. Here are the three pricing models, the four levers that move your quote, and how to run the evaluation without wasting a quarter.

Read article

Put it into practice

Connect your warehouse read-only and see your lineage map in 15 minutes. Transparent pricing, no card to start.