Testing data like you test code

Silent failure is the expensive kind

A pipeline that crashes gets fixed the same morning. A pipeline that quietly loads half the rows gets discovered six weeks later by someone in finance, after decisions have been made on the numbers.

Data tests exist to convert the second kind into the first.

Four tests that catch most of it

Freshness. Did this table update when it should have? The cheapest test to write and the one that catches the most real incidents.

Row count deltas. Today’s load is 40% smaller than yesterday’s. Occasionally legitimate, usually not.

Uniqueness on the key. Duplicate primary keys quietly inflate every aggregate downstream, and the totals still look plausible.

Referential integrity. Orders pointing at customers that do not exist mean a join is about to drop rows nobody will notice.

Run them where the work happens

Tests in a separate quality tool get out of date. Put them in the same project as the transformations, run them on the same schedule, and fail the pipeline rather than warn.

models:
  - name: fct_orders
    tests:
      - dbt_utils.recency: { datepart: day, field: ordered_at, interval: 1 }
    columns:
      - name: order_id
        tests: [unique, not_null]

Route alerts to someone who can act

An alert in a channel nobody reads is worse than no alert, because it creates the impression of coverage. Name an owner per domain and send failures there.

Start with the tables people argue about

You do not need coverage everywhere on day one. Instrument the five tables that feed the reports people already distrust, and let the results make the case for the rest.