# Python client and CLI

`dataq-client` is a Python package and a `dataq` command for driving DataQ from scripts, CI
pipelines, notebooks and schedulers. Its most common use is gating a deploy on a data-quality
result: trigger a suite, wait for it, and fail the pipeline step if a check failed.

```bash
# A released version: the wheel attached to the DataQ release matching your server's version
VERSION=1.2.0   # your DataQ server's version
pip install "https://github.com/TheurgicDuke771/DataQ/releases/download/v$VERSION/dataq_client-$VERSION-py3-none-any.whl"

# The latest from main
pip install "git+https://github.com/TheurgicDuke771/DataQ.git@main#subdirectory=packages/dataq-client"

# The latest from main, editable (to track or change the client locally)
git clone https://github.com/TheurgicDuke771/DataQ.git && pip install -e DataQ/packages/dataq-client
```

The client is not published to PyPI. Each [DataQ release](https://github.com/TheurgicDuke771/DataQ/releases) from 1.2.0 on carries its wheel
and sdist as release assets; install the one matching your server's version. Until your server's
release carries one, install from `main`.

It authenticates with a [personal access token](api-keys.md), which acts as the user who minted
it, with the same workspace role and per-suite grants.

## Gate a pipeline step on a suite

```bash
export DATAQ_URL=https://dataq.example.com   # the host that serves the DataQ web app
export DATAQ_PAT=dq_live_...                 # from your CI secret store

dataq run 7c1e…-suite-id --wait
```

`--wait` polls the run until it finishes and exits with a code your pipeline can act on:

| Exit code | Meaning |
|---|---|
| 0 | The run finished and nothing failed |
| 1 | The worst result was a **warning** |
| 2 | A check **failed** (`fail` or `critical`) |
| 3 | The run itself did not complete (failed or cancelled), or a check could not be evaluated |
| 4 | A client, authentication or network problem: the result is unknown |

A GitHub Actions step:

```yaml
- name: Data-quality gate
  run: |
    pip install "https://github.com/TheurgicDuke771/DataQ/releases/download/v$DATAQ_VERSION/dataq_client-$DATAQ_VERSION-py3-none-any.whl"
    dataq run "$SUITE_ID" --wait
  env:
    DATAQ_URL: ${{ vars.DATAQ_URL }}
    DATAQ_PAT: ${{ secrets.DATAQ_PAT }}
    DATAQ_VERSION: 1.2.0   # your server's version
    SUITE_ID: 7c1e…
```

The token is read from `DATAQ_PAT` only, never from a command-line flag, where it would land in
shell history and process listings. To treat warnings as a pass, allow exit code 1 in your
pipeline (for example `dataq run … --wait || [ $? -eq 1 ]`).

### Gate a pipeline between its own stages

A suite run gates one suite. `dataq gate` gates a **pipeline**: it runs every suite bound to
the pipeline (Suite → Triggers) for this pipeline run, waits, and exits `0` passed, `2` failed,
`3` error or `4` timed out or unreachable (see
[Gate a pipeline on DataQ](orchestration.md#gate-a-pipeline-on-dataq)):

```bash
dataq gate --provider dbt --pipeline nightly_build --env prod --run-id "$RUN_ID"
dataq gate --provider airflow --pipeline load_finance --env prod --run-id "$RUN_ID" \
  --fail-on warn --no-trigger     # only report on runs the success event already started
```

Repeating the command for the same run id never starts a second run.

Other commands:

```bash
dataq wait <run-id>                               # wait for a run someone else started
dataq run <suite-id> --wait --json                # the outcome as JSON
dataq export <suite-id> -o orders.yaml            # the suite's portable document (.json works too)
dataq import orders.yaml --connection <conn-id>   # create a NEW suite from it
dataq validate orders.yaml --connection <conn-id> # would it be accepted? exit 2 if not
dataq drift orders.yaml --suite <suite-id>        # does the suite differ from the file? exit 2 if so
dataq apply orders.yaml --suite <suite-id>        # make the suite match the file (--prune deletes extras)
```

## From Python

```python
from dataq_client import DataQClient

with DataQClient() as dq:            # reads DATAQ_URL / DATAQ_PAT, or pass them in
    run = dq.trigger_run(suite_id)
    outcome = dq.wait_for_run(run.run_id, timeout=1800)

if outcome.status != "succeeded" or outcome.worst_severity in ("fail", "critical"):
    raise SystemExit(f"data quality gate failed: {outcome}")
```

A run's outcome has **two separate answers**, and the client never merges them into one
"passed" flag:

- **`status`** says whether the run itself finished: `succeeded`, `failed` or `cancelled`.
- **`worst_severity`** is the worst check result: `warn`, `fail`, `critical`, or `None` when
  nothing failed.

A run can **succeed** and still report a critical failure: every check ran, and one found bad
data. It can also **fail** with no results at all, when the target could not be read. Check
both.

`wait_for_run` decides a run has finished from its `status` alone. The pass counts of a run
that is still going describe only the checks done so far. It polls every 5 seconds at the
fastest, backing off to 30. When the API rate-limits the token, it raises `RateLimitedError`
instead of retrying in a loop.

## The rest of the API

`DataQClient` covers running, waiting and gating, exporting, importing, validating and applying
suites, and acknowledging or resolving incidents. Every other REST endpoint is available through the generated layer, typed
from the API's own specification:

```python
from dataq_client.generated.api.suites import list_suites

for suite in list_suites.sync(client=dq.api) or []:
    print(suite.id, suite.name)
```

## Versions

The client is released with DataQ: client `1.x.y` is generated from server `1.x.y`. Use the
client version that matches your server. [API compatibility](../reference/api-compatibility.md)
describes what stays stable across upgrades.