Markdown

Recommended usage¶

How to activate each DataQ feature and the recommended way to configure it. This is the setup-time companion to Features (what exists) and Best practices (the run-time "signal, not noise" philosophy) — follow it top to bottom the first time you set up a workspace.

Each step is Do (how to turn it on) → Recommended (the way that pays off later).

1. Connect a datasource¶

Do: Connections → Add connection → pick the type → fill the spec-driven form → Test.

Recommended:

Use a least-privilege, read-only credential scoped to what the checks read.
Snowflake: prefer key-pair auth over a password; use a dedicated role/warehouse.
Create one connection per environment (snowflake-dev, snowflake-prod) so a suite's env is unambiguous and promotion is a re-point, not a rewrite.
ADF / Airflow / dbt are orchestration providers, not datasources — add them here too, but you don't write checks against them (see step 5).

2. Author a suite and its first checks¶

Do: Suites → New suite → bind a connection + target (a table, or a file/batch pattern) → add checks in the editor.

Recommended:

One suite = one target = one env. Name it for what it watches (orders — snowflake prod).
Start with freshness + volume, not value-level rules — most real incidents are "the load didn't run" or "half the rows are missing". Add a freshness monitor on the load/updated timestamp and a volume monitor before anything else.
Use the column profiler and a dry-run to see today's numbers before writing a value check, so you set it against reality.
Reach for Custom SQL only when a rule spans columns or needs a join — prefer a catalog expectation when one exists (cheaper, better samples). See Custom SQL best practice.

Each check is classified by DQ dimension automatically (a not-null check is Completeness, a freshness monitor is Timeliness). Leave it unless the check means something else — the same range check is Validity when it bounds a percentage and Accuracy when it asserts a reconciled total. Custom-SQL checks have no derivable dimension, so set one yourself or leave them unclassified; the asset scorecard counts them separately rather than filing them under a dimension they may not belong to.

3. Set severity thresholds that mean something¶

Do: on each check, set warn / fail / critical thresholds (or leave blank for binary pass/fail).

Recommended: band from the dry-run / profiler baseline — set warn just above today's observed unexpected-%, fail at an actionable breach, critical only for page-worthy. The check should be green on day one and loud only when reality changes. Full rationale: Severity best practice.

4. Protect failing-row samples (column policy)¶

Do: on the suite, open the column policy and accept the classifier's Auto-detect suggestion (or set the identifier + PII columns by hand).

Recommended: always set a policy on suites over tables that can contain personal data — it keeps non-sensitive breaches debuggable while masking PII in samples. If you repoint the suite to a new target, re-run Auto-detect (DataQ won't clobber your policy automatically; it logs suite_policy_possibly_stale). See Protect the samples.

5. Automate runs — triggers first, schedules second¶

Do (trigger): register the orchestrator connection → add the provider's webhook (Admin → Integrations shows the URL) or the callback snippet (integrations/) → on the suite, Triggers → bind (provider, pipeline/DAG/job, env).

Do (schedule): suite → Schedules → a 5-field cron + timezone.

Recommended: if an orchestrator produces the data, trigger on its success — it knows when the data is ready and correlates results to the pipeline run. Schedule only what has no orchestrator (e.g. flat-file drops), at a cadence matched to arrival, not "every 5 minutes to be safe". See Prefer triggers over schedules.

6. Configure alerting¶

Do: Suite → Notifications → pick channels (Teams / Slack / email), the threshold, and recipients. Store channel webhooks/keys as secrets (Admin → Integrations for the inbound ones).

Recommended:

Keep the default warn-and-worse threshold; drop to fail-only for noisy exploratory suites rather than turning alerts off.
Snooze a known-broken check during an incident instead of deleting it — the history and the re-fire on expiry are the point.
Trust dedup: a red suite that's quiet is dedup working; the Results page is ground truth.

See Alerting hygiene.

7. Wire an AI assistant (MCP)¶

Do: mint a PAT (Profile → API keys), then add DataQ's /mcp server to your client (Claude Desktop / VS Code Copilot / Cursor) with the PAT as the bearer — full walkthrough in AI assistants (MCP setup).

Recommended: give the assistant a scoped PAT (a member, not an admin, unless it needs workspace-wide reach), set a sensible expiry, and rotate it like any credential. The assistant acts as that user — it sees exactly what the user can.

8. Set up access¶

Do: users sign in via SSO in production (email OTP is the local/eval default — ADR 0032). The first admin is seeded by the WORKSPACE_ADMIN_EMAILS bootstrap allowlist; from then on manage roles under Admin → Members. Share individual suites (view / edit) from the suite.

Access is two independent axes (ADR 0033):

	What it gates
Workspace role — `admin` / `member` / `viewer`	Whole resource classes. Connection create/edit/delete/re-auth is Admin-only; Viewers are read-only everywhere.
Per-suite grant — `view` / `edit`	One suite at a time. A Viewer is capped at `view` even if granted `edit`.

Recommended:

Share by default with view; reserve edit for the owning team.
Keep the Admin role minimal — an Admin can read every suite's results, including failing-row samples (the one place PII lands), manage or delete any suite (ADR 0027), and manage the connections every suite runs on. For a regulated/PHI deployment, treat the access-audit trail as a prerequisite before granting it broadly.
Give day-to-day authors member. They can build suites and checks against existing connections without holding the credentials for them.
WORKSPACE_ADMIN_EMAILS only ever grants — it is a bootstrap seed and a lockout break-glass, never a way to demote. Removing someone from it does not take their Admin role away; change the role in Admin → Members.
Role changes take effect immediately, on the next request — including for PATs the user already holds. There is no token to revoke.

Once this is in place, the Best practices page covers the ongoing operating discipline that keeps the alerts meaningful.