Column-level lineage

​

Supported sources

Column-level lineage works for every supported data warehouse and relational database, and continues down to Looker and Tableau.

​

Visibility into issues

Column-level lineage can be inaccurate in two cases. In both cases Coalesce Quality marks the lineage as uncertain.

​

Parsing of SQL failed

If the parser cannot read the SQL query, the Code & Changes tab of the asset and the lineage graph show an error.

​

Extraction of dependencies failed

If the SQL contains unusual constructs, the column flow can be resolved incorrectly. Coalesce Quality validates the result after parsing, and shows the validation results in the Code & Changes tab of the asset and on the lineage graph.

​

Limitations

If a SQL construct you use is not supported, contact support.

​

BigQuery STRUCT literal lineage

Coalesce Quality tracks lineage of individual STRUCT fields and of the RECORD column itself, so lineage works when you use them in CTEs inside one SQL statement. Between two assets, lineage shows only columns, not RECORD fields.

Consider this example:

WITH c AS (
  SELECT STRUCT (c1 AS a, c2 AS b) AS field FROM `prj`.`ds`.`tbl`),
    d AS (SELECT STRUCT (field.a as d) double_nested
          FROM c)
SELECT  double_nested.d
FROM d

Here, lineage goes from the output column d to the column c1 in prj.ds.tbl.

Now take two assets. The first is defined as

CREATE VIEW ds.rec_tbl(r) AS
  SELECT STRUCT (c1 AS a, c2 AS b) AS r FROM `prj`.`ds`.`tbl`

The second is defined by this query:

SELECT r.a AS r FROM `prj`.`ds`.`rec_tbl`

Lineage shows that column r depends on both c1 and c2.