Running checks from your pipeline

Run on-demand monitors and SQL tests from your pipeline when new data lands, and get the results back in the same call

A scheduled check runs at a fixed time, whether the data is ready or not. If your pipeline knows when a load is done, it can run the checks itself at that moment, with one API call. The call waits for the checks to finish and returns their results, so the pipeline can stop when a check fails.

Which checks run

The call runs the checks on the tables you name:

• Monitors with the schedule set to on demand. In YAML, this is schedule: ondemand.
• SQL tests with the schedule set to On demand.

Checks with a schedule do not run from the call. They keep their own schedule.

Create an API client

Create an with the Trigger Data Checks scope (SCOPE_DATACHECKS_TRIGGER). Exchange its credentials for an access token as described in Fetching Access Token.

Trigger the checks

Send POST /api/datachecks/scheduler/v1/trigger with the tables in entity_ids. This example runs the checks on one Snowflake table:

Each warehouse has its own identifier type, for example bigquery_table or databricks_table. See the API reference for the fields of each.

Coalesce Quality runs up to 10 checks at the same time. The response has one result for each check that ran.

Results

Each result has a status:

Status	Meaning
DATACHECK_STATUS_PASSED	The check passed.
DATACHECK_STATUS_FAILED	The check found a problem, for example a SQL test returned rows or a monitor found an anomaly.
DATACHECK_STATUS_ERROR	The check itself could not run, for example because of a SQL error. message has the reason.
DATACHECK_STATUS_SKIPPED	The check did not run.

A monitor is skipped when it has already checked the current time interval. For example, a monitor with a daily time partition checks one day at a time. If you call twice on the same day, the second call skips it. A monitor is also skipped when its warehouse integration is disabled.

A monitor result has one prediction for each segment, with the measured value and the expected value. A SQL test result has rows_count, the number of rows the test query returned.

Calls that come too close together

If you send a call for the same set of tables within a few seconds of the previous one, the call is rejected with HTTP status 429 and the message request throttled, retry after N seconds. Wait the given number of seconds, then retry.