Markdown

Security & data handling¶

How DataQ authenticates users, stores secrets, handles the data it sees, and what it keeps. This is a plain-language overview for people evaluating or operating DataQ. It is not a legal compliance certification — much of GDPR/HIPAA is organizational (DPAs, BAAs, consent, lawful basis) and is the deploying organization's responsibility.

Authentication & access¶

Single sign-on (OIDC). Users sign in through your identity provider; DataQ is provider-neutral (validated against Azure AD and AWS Cognito). The backend validates the token on every request — there is no local password store.
Who may hold an account is your decision, not your IdP's. DataQ provisions a user on first successful sign-in, so without a gate the identity provider's registration policy silently becomes DataQ's access policy. That is fine for an invite-only tenant and wrong for a provider with self-service sign-up. Two independent controls:
Keep the identity provider invite-only (the AWS reference stack sets allow_admin_create_user_only on the Cognito pool; an Azure AD tenant is invite-only by nature).
Set OIDC_ALLOWED_EMAILS / OIDC_ALLOWED_DOMAINS as the app-side allowlist. It is checked on every request, not only at first sign-in, so removing an entry revokes an existing account rather than merely blocking a new one.

Leaving the allowlist empty means "admit anyone the issuer vouches for" — deliberately, so that upgrading DataQ can never lock an existing workspace out — and the API logs auth_oidc_no_signup_allowlist at WARNING on every boot in that state. If you see that line, confirm your IdP is invite-only. - Personal access tokens (PATs). For headless / AI-client use, users mint dq_live_ tokens (API keys). Tokens are hashed (SHA-256) at rest — the plaintext is shown once and never stored — and carry the same authz as the user, on REST and MCP. They can be scoped with an expiry and revoked. - Email one-time codes (OTP). For deployments with no identity provider, a human can sign in by proving they own a mailbox: DataQ mails a 6-digit code, and verifying it sets an HttpOnly, SameSite=Lax session cookie (fixed lifetime, default 24 h, no refresh token — signing in again is the refresh). Logout revokes the session server-side, and both expiry and revocation are checked on every request. Sessions are hashed (SHA-256) at rest, like PATs. Sign-up is allowlist-only — there is no open registration. MCP does not accept a session: it is a browser credential, and PATs remain the headless credential (ADR 0032). Secure is inferred, not unconditional: by default the backend reads the X-Forwarded-Proto the proxy forwards, and marks the cookie Secure only when that says https. That is only as trustworthy as the proxy. The reference frontend nginx forwards the edge's header and falls back to its own $scheme only on a direct connection (it used to overwrite the header with its own plaintext upstream scheme, which dropped Secure on live HTTPS), so inference is correct on that stack. If you front DataQ with your own proxy, confirm it forwards the real client-facing scheme — or simply set AUTH_SESSION_COOKIE_SECURE=true, which any HTTPS deployment can do and which stops consulting the header at all. Leave it unset only for genuinely plain-HTTP local dev: a hard-coded Secure there fails silently (the browser accepts the Set-Cookie and then never sends it back). Before turning OTP on for real users, run the admin-gated SMTP pre-flight test — POST /api/v1/admin/auth-email/test sends a real message to the caller's own address over the configured AUTH_EMAIL_* transport and, on failure, reports exactly which stage broke (connect / tls / auth / send) rather than a generic error. It never accepts a recipient argument — it can only mail the admin who called it, and that address is echoed back in the response by design (it's the point of the check). The SMTP password never appears in the response or in any log line; the recipient address never appears in a log line either, only in the response, to the same admin who supplied it by calling the endpoint. Transport options: AUTH_EMAIL_TLS_MODE picks starttls (default) / implicit (SMTPS on :465) / none (plaintext — test-only, loudly warned on every send, and there is no way to disable that warning), and AUTH_EMAIL_CA_BUNDLE names a PEM the mailer trusts for a relay on a private CA. The bundle is scoped to the mailer's own connection only — it never widens or replaces the trust store any other TLS client in the process relies on (Key Vault, Snowflake, ADLS, webhooks). There is deliberately no option to skip certificate verification: a private CA bundle is judged to cover the legitimate case, and disabling verification was rejected as a shortcut around it rather than shipped as a convenience. It is capped per admin (ADMIN_EMAIL_PREFLIGHT_PER_10MIN, default 3 per 10 minutes), because every call is a real connection to your mail relay: admin-gating bounds who can open one, not how many, and a scripted or compromised admin token could get your sending account throttled or blocked by the relay — a worse outage than the misconfiguration the pre-flight exists to catch. Over the cap is a plain 429; the cap fails open if the counter store is unavailable, and 0 disables it.

Email as the root of trust (read this before enabling OTP)¶

Under email OTP, the mailbox is the credential. The consequences are not subtle, and they are the reason this mode is opt-in rather than the default:

Mailbox compromise is account compromise. Anyone who can read the address's inbox can sign in as that person and see every suite they own or are shared on. There is no second factor to fall back on.
Mailbox compromise of an admin is workspace compromise. If the address is on WORKSPACE_ADMIN_EMAILS, that access is workspace-wide — including every suite's failing-row samples, the one place PII can appear.
One user row per normalized email, deliberately. An OTP sign-in whose address matches an existing SSO-provisioned user resolves to that user, so grants, shares and PATs never fragment across authenticators. The flip side: in a deployment running both SSO and OTP, an emailed code is an alternative route into an SSO identity. If your IdP enforces MFA and your mail does not, OTP is the weaker of the two doors — decide that deliberately, and consider keeping the allowlist to addresses that have no SSO identity.
Mitigations DataQ does apply: codes expire in 10 minutes, are single-use, allow at most 5 verification attempts, and a new request invalidates the previous code; requests are capped per mailbox as well as per IP; the sign-in endpoints return an identical response body and status whether or not an address is known, so the response content cannot be used to enumerate who has an account. The code-request endpoint also holds every such response to a common minimum latency (AUTH_OTP_REQUEST_MIN_SECONDS, default 1s), so the code-mint an eligible address incurs no longer stands out against an ineligible one's in-memory lookup. The mail itself is not sent on the request path at all (ADR 0032 §7 as amended): the api mints the code and hands delivery to the worker, which does the SMTP password lookup and the send. A floor can pad an ineligible address up to a slow relay but never an eligible one down, so a synchronous send slower than the floor used to re-open the channel — a relay's latency now cannot reach the response at all. A mail or broker failure is an operator log line (otp_send_task_failed on the worker, otp_request_dispatch_failed on the api) and the response stays ok, because the earlier 502/503 answered differently for members than for strangers while the relay was down. Use the admin SMTP pre-flight to find a broken relay; it is still synchronous by design. What the floor does not cover, stated plainly: a database round trip or a broker publish slower than the floor still overruns it, and setting the floor to 0 removes it. Code verification carries the same floor on its own setting (AUTH_OTP_VERIFY_MIN_SECONDS, default 0.5s): every rejected code answers an identical 401, but an address that has a live code outstanding does more database work than one that has none, so the rejection's timing used to reveal which — and cheaply, because a code is minted only for an allow-listed address, so a single request-a-code call against the address being probed (which answers identically either way, revealing nothing itself) is what sets that difference up. Every 401 is now held to the same minimum. A successful verification is deliberately not held (a caller who knows the code learns nothing from that), a database round trip slower than the floor still overruns it, and 0 removes it here too. Treat both floors as raising the attacker's cost from a handful of samples to a statistical exercise, not as a constant-time guarantee.
Mitigations that are yours: MFA on the mailbox, a mail domain with SPF/DKIM/DMARC, and a minimal allowlist. If you have an IdP, prefer SSO — OTP exists for the case where you do not.
Per-suite authorization. Access is granted per suite (view / edit); a caller only ever sees suites they own or are shared on. There are no ambient "see everything" reads except the Admin role below.
Workspace membership decides who is in at all. An Admin admits and removes addresses in Admin → Members. While that list is empty nothing changes: who may sign in is decided entirely by the allowlist settings below, exactly as before. Once it is non-empty, a person is admitted if the list names them or an allowlist does — the allowlists are grant-only and can never remove somebody the list admits. Removing a member takes effect on their next request for every credential kind: an identity provider sign-in, a live browser session, and every API key they hold. Membership does not create accounts at your identity provider; that stays a prerequisite.
Workspace roles — Admin / Member / Viewer. A stored users.role (ADR 0033), not just an env allowlist. Admin has workspace-wide visibility over every suite, its results, and schedules; can revoke any per-suite share, transfer any suite to another member or admin, and delete any suite (each audited with an admin-override flag and, for delete, the exact counts destroyed); and is the only role that can create, edit, delete or re-auth a connection — a Member can reference, test and run against an existing connection but cannot mutate or re-credential it (closes the earlier hole where any authenticated user could delete or re-point the Snowflake connection every suite ran on). Testing a saved connection is deliberately Member+ rather than Admin-only: authoring a suite against a connection you cannot verify is not a workable flow, and a test reveals nothing a Member could not already learn by running a suite. Testing an unsaved draft is Admin-only, because that request carries caller-supplied config rather than config an admin stored. Viewer is capped at view everywhere, including on any share it receives, and cannot test a connection — the probe opens an outbound connection using stored credentials, which a read-only tier has no reason to trigger. The Viewer cap is enforced at the point of use, not only when a share is granted, so demoting someone to Viewer immediately downgrades any edit share they already hold. Because Admin visibility includes failing-row samples (the one place PII can appear), grant it sparingly and treat a data-access audit trail as a prerequisite before granting it in a regulated deployment. WORKSPACE_ADMIN_EMAILS still resolves to Admin as a bootstrap / break-glass path for a fresh or locked-out workspace — keep that allowlist minimal, and prefer in-app role management (Admin → Members) once at least one Admin exists. It only ever grants: removing an address never demotes anyone, so demotion has exactly one route, where the guard runs.
An env-level actor can always mint an Admin. This is the deliberate cost of keeping a break-glass path, and it is worth stating rather than leaving implicit: anyone who can set environment variables on the API container can make themselves a workspace admin, without a role change, an audit line, or anyone's approval. Treat write-access to the API's environment as equivalent to workspace-admin when you do access reviews. In steady state, leave the allowlist empty.
You cannot remove the last Admin. A role change must leave at least one stored-role admin; allowlist-resolved admins deliberately do not count toward that (the env entry can vanish on the next deploy, which would leave the workspace with no admin and no in-app way to mint one). Promote a successor first. Role changes are audited durably: every change writes an append-only audit_events row (user.role_change, old and new role, actor, request_id) inside the same transaction as the change, queryable by a workspace admin at GET /api/v1/admin/audit-events (ADR 0041) — alongside the structured log line. Every row is hash-chained to the one before it, so a direct database edit or deletion outside the app is detectable (see Compliance below).

Network exposure¶

The frontend is the only public surface; the API runs on internal ingress and is reached only through the frontend's same-origin /api, /healthz, and /mcp proxy (ADR 0028 §5). All traffic is over HTTPS/TLS.
The MCP AI-assistant endpoint is fail-closed — unauthenticated requests are rejected (ADR 0008), and it is not mounted at all unless the deployment has a working sign-in configuration. It authenticates from the Authorization header only and never reads a cookie, so a signed-in browser lured to a hostile page cannot be used to drive it. In an OTP-only deployment there is no identity provider, so a PAT is the only credential MCP accepts: a session token is rejected by prefix before any validation, and a bearer that is neither is rejected outright rather than handed to an unconfigured validator — the absence of a JWT verifier is a refusal, never a skipped check.
Browser security headers. Every response from the frontend carries a Content-Security-Policy (default-src 'self', script-src 'self', object-src 'none', frame-ancestors 'none', base-uri 'self'), HSTS, X-Frame-Options, X-Content-Type-Options: nosniff, Referrer-Policy and Permissions-Policy. The CSP is the backstop that matters here specifically because the UI renders values that came from your warehouse — failing-row samples, error text, custom SQL. connect-src is configured per deployment (DATAQ_CSP_CONNECT_SRC) because the sign-in flow talks directly to your identity provider; the shipped default (https:) is permissive so that upgrading the image cannot break an existing sign-in, and the reference stacks narrow it to their exact IdP origins.
Edge rate limiting (AWS). In addition to the in-app limiter below, the CloudFront distribution carries a WAF per-IP rate ceiling. The two are not redundant: the app limiter is per-token, understands DataQ's request classes, and fails open if its Redis store is unavailable — so it is deliberately biased toward availability. The WAF rule sheds a flood before it reaches the application at all.
CSRF. The session cookie is SameSite=Lax, which blocks cross-site POSTs; that only holds while every state-changing endpoint is a POST/PATCH/PUT/DELETE, so the test suite audits the whole route table for a GET that mutates. Sign-in and sign-out are both POST-only, and the SPA and API share an origin through the proxy, so a cross-site request can neither carry nor read the cookie.
Two coordinated auth-mode selectors. The frontend's runtime DATAQ_AUTH_MODE and the backend's inferred mode (SSO variables, or the OTP mailer + allowlist block) are separate contracts — neither can derive the other. Set them together; the full table is in the repository's deploy/README.md and .env.app.example. The backend refuses to start on a half-configured OTP block rather than come up unable to log anybody in.

Secrets¶

Datasource credentials, webhook signing keys, and channel secrets are held in a secret store behind a seam — Azure Key Vault in the primary reference deployment, AWS Secrets Manager on the AWS stack, OpenBao/Vault self-hosted — never in the database or in git. The app reads them via a managed identity (Azure) or the task IAM role (AWS).
Secret references (names), not secret values, are stored alongside connections. Deleting a connection removes its secret (soft-delete on Key Vault).
A reference is server-owned: a client may echo back the one already stored on a connection, but may never introduce or repoint one. Otherwise a caller could name someone else's secret and have the server resolve it on their behalf.
A stored credential is never sent to a destination the caller changed. Editing a config field that decides where a credential goes — Snowflake account, ADLS account_url, S3/dbt endpoint_url, Unity Catalog workspace_url, Iceberg catalog_uri/warehouse/ properties/secret_property, Airflow base_url, dbt artifacts_uri — requires re-supplying that credential in the same request, or the update is rejected (422 credential_redirect). Moving a connection to a new host is a supported operation; doing it with a credential you do not know is not. This is why an Admin, who may rotate a credential, still cannot read one.
Inbound webhooks are authenticated: ADF by a shared secret, Airflow and dbt by an HMAC-SHA256 signature keyed on a stored signing key.
A daily orphan-secret sweep reconciles the store against every row that should own a secret, reporting (never purging, unless SECRET_ORPHAN_PURGE is explicitly set) any entry no row references. The last run — beat or an admin's on-demand GET/POST /admin/secret-sweep(/run) — is recorded with secret names only, never values; a store outage records a classified error rather than a fake "0 orphans found", and the on-demand run always stays report-only regardless of the purge setting.

The data DataQ sees, stores, and redacts¶

DataQ runs checks against your data; it is not a copy of your data. What it persists:

Metadata — suites, checks, connection config (no secrets), schedules, trigger bindings.
Results — per-check pass/fail + a numeric metric_value, and for failing checks a small failing-row sample.
Failing-row samples are the one place results can carry PII/PHI. They are redacted at the boundary, column-aware: a suite's column policy (auto-derived by a classifier or set by hand) keeps non-sensitive breach values debuggable while masking PII columns to <redacted>. The numeric counts and row/column shape are kept.
Zero-sample privacy mode (PRIVACY_ZERO_SAMPLE_MODE=true) is a deployment-level step past redaction: no failing-row sample is ever persisted at all — not full, not redacted — only aggregates (metric_value, unexpected counts/percents). It gates the one write-path choke point every check kind funnels through, so results, alerts and MCP all inherit the suppression from what's actually in the database. It is also a workspace setting: an admin can turn it on from Admin → Settings → Privacy & failing samples, and the change applies on the next run with no restart, because every sample writer reads one resolver per request. The environment variable is the floor, not the whole answer — the effective state is the env value OR the stored toggle, so a deployment that pins it on in the environment cannot be turned off from the app (the toggle is shown pinned). GET /admin/deployment reports the effective value and which of the two turned it on. Turning it on does not delete samples already stored; the retention sweep does.
Logs & traces are PII-redacted at the logger level, and secret values never enter them.

Retention¶

Failing-row samples are purged after a retention window (PII-minimisation), while the aggregatable metric_value history is kept for trends and baselines — so you lose the raw rows but keep the signal.

Encryption¶

In transit: HTTPS/TLS on every public surface, and TLS to PostgreSQL (sslmode=require) and to the AWS cache (rediss://). Two internal hops are plaintext, both deliberately and both stated rather than glossed: the Azure frontend→api hop, which never leaves the Container Apps environment; and — on AWS only — the CloudFront→ALB origin hop, because that stack has no custom domain and therefore no certificate the load balancer could serve. The second one does cross the AWS network, so it is a real, tracked gap rather than a contained one.
At rest: PostgreSQL, the object stores, and the secret store (Key Vault / AWS Secrets Manager) encrypt at rest in both reference deployments. The AWS cache (ElastiCache — the Celery broker and rate-limit counters, not a data store) currently does not.

At-rest encryption per resource — the security-review evidence¶

What encrypts what, with which key, and where the vendor says so. This exists because "encrypted at rest" is only useful to a reviewer with the mechanism attached; each row below is checkable against a first-party citation rather than our assertion.

Azure (primary reference deployment)

Resource	Encrypted at rest	Key	Citation
PostgreSQL Flexible Server — user + system databases, server logs, WAL segments and backups	Always, unconditionally	Service-managed (Azure-managed) by default	Data encryption at rest
Key Vault (the `SecretStore` — warehouse credentials, webhook URLs)	Yes	Microsoft-managed, FIPS 140-validated HSM-backed	About keys
Container Apps images / revisions (GHCR-sourced)	Registry-side	Provider-managed	—
Log Analytics + Application Insights (telemetry, PII-redacted at the logger)	Yes	Microsoft-managed by default	Azure Monitor data security

AWS (second reference deployment)

Resource	Encrypted at rest	Key	Notes
RDS PostgreSQL	Yes — `storage_encrypted = true`, asserted in our own IaC	AWS-managed KMS	The one row our Terraform/OpenTofu actually asserts, because that stack owns its database
Secrets Manager (`dataq/conn-*`)	Yes	AWS-managed KMS
S3 (landing bucket)	Yes	SSE-S3 default
ElastiCache (Celery broker + rate-limit counters)	No	—	Not a data store, but the asymmetry beside an encrypted RDS is exactly what a review flags

Customer-managed keys (CMK)¶

Data at rest is encrypted with service-managed keys by default on both clouds. CMK — where you hold the key — is not wired into the IaC in this repo, and the reason splits into one product constraint and a set of deployment prerequisites.

The product constraint: on Azure, CMK is creation-time-only. Microsoft is explicit — "You can select the mode only at server creation time. You can't change the mode from one to another for the lifetime of the server." The only route to CMK on an existing server is restoring a backup onto a new one, so it is a data migration rather than a Terraform toggle, and it is one-way: reverting to service-managed keys requires another restore. (Limitations of CMK)

The prerequisites, which a deployment either meets or does not:

Your IaC must own the database server from creation. A stack that attaches to a pre-existing server has no server resource to hang a key on.
The key vault must be non-destroyable. If the vault holding a CMK is deleted, the server goes Inaccessible and denies every connection — so purge protection must be on, which is irreversible once enabled. Azure additionally requires 90-day deleted-vault retention, a value that cannot be changed after the vault is created: a vault set lower needs a new vault, not a setting change.
You must commit to key rotation operationally. A key that expires, is disabled, or becomes unreachable takes the database offline within about an hour.

Meeting all three is a deployment topology, not a feature flag — which is why CMK is documented here rather than shipped as an option that could be enabled unsafely. A stack that provisions its own server and a purge-protected vault can adopt it; the maintainers' reference deployment does neither, so it is out of scope there.

Column classification from your warehouse¶

DataQ masks failing-row samples by default and surfaces only what it can justify showing. The strongest justification available is your own governance: if a column is classified in the warehouse, DataQ reads that classification and treats it as the floor — a suite-level setting cannot lift it.

The convention¶

A fixed, documented convention, not a per-connection mapping. A mapping would be more flexible and worse: every deployment would express the same idea differently, the mapping itself would become an unreviewed security control, and a typo in it would silently un-mask a column.

Tag your columns with the key dataq_classification (matched case-insensitively):

Value	Effect
`sensitive` · `pii` · `confidential` · `restricted` · `secret`	Always masked. A suite policy cannot show it
`public` · `non_sensitive` · `nonsensitive`	Cleared — may be shown, unless the value itself is affirmatively personal data
anything else	Ignored, exactly as if untagged — see below

Snowflake additionally honours its own PRIVACY_CATEGORY, set by Snowflake's built-in classification. Its values (IDENTIFIER, QUASI_IDENTIFIER, SENSITIVE) all mean personal data, so all three mask. It has no clearing side: Snowflake omits the tag for data it does not consider personal rather than marking it public, so an unexpected value means unknown, not safe.

An unrecognised value is ignored rather than guessed at. Guessing has only two directions, and one of them un-masks data. internal is the worked example: it sounds like a classification and is commonly the default stamp on everything, so reading it as a clearance would clear whole tables in exactly the organisations careful enough to tag them.

Inherited tags are not read. A tag applied to a table or schema is reported by the warehouse against every column beneath it; DataQ accepts only tags applied to the column itself. An inherited tag is a statement about the container, and reading one as a per-column clearance would clear a whole schema from a single misplaced public.

The tag name is matched without its namespace, and that is a constraint on you rather than a feature: DataQ honours a tag named dataq_classification wherever it lives, because knowing which database or schema should be authoritative would require per-deployment configuration, and this convention is deliberately fixed. Do not create tags with this name for any other purpose — a same-named tag elsewhere carrying a public-family value would be honoured as a clearance.

Who can apply the tags, and what DataQ needs to read them¶

Two different privileges, and conflating them is the usual stumbling block.

Applying a tag is a data-steward action, and on Snowflake it needs more than the CREATE TAG grant that creates the tag object. Either:

APPLY TAG on the account — the global route; or
APPLY on the tag and OWNERSHIP of the object being tagged.

A role that can create the tag but does not own the table is refused. Verified against a live warehouse, not inferred — though our run held both halves of the second route at once (the steward created the tag and owned the table), so it confirms the combination rather than isolating which half was doing the work.

Related, if your connection uses one: a role-scoped programmatic access token cannot switch to another role, even one the underlying user holds. A PAT issued for a read-only role stays read-only no matter what the user is granted.

Reading them needs nothing extra. DataQ queries INFORMATION_SCHEMA.TAG_REFERENCES_ALL_COLUMNS (Snowflake) and information_schema.column_tags (Unity Catalog) as the connection's own role — no ACCOUNT_USAGE grant, no elevated identity. That is deliberate: ACCOUNT_USAGE lags by up to two hours, and a stale classification is the failure this feature exists to prevent.

That holds for PRIVACY_CATEGORY too, which is worth stating because the tag lives in the shared SNOWFLAKE database and a reasonable reader would expect it to need IMPORTED PRIVILEGES. It does not: verified live against a connection role holding no such grant, which nonetheless saw the tag and all three of its values. The check mattered because failure here would be silent — a role that cannot see a tag gets zero rows, not an error, so the column would look untagged and fall back to the name heuristic. If you restrict tag visibility in your own account, confirm your DataQ connection role can still read both tags.

Where it applies¶

Only Snowflake and Unity Catalog have a column-tag concept. ADLS, S3, Iceberg and flat files have no authoritative source to read, so for those the classification remains the suite's own policy, the name/value classifier, and fail-closed mode. This is a limit of the platforms, not a gap in the implementation, and it is stated here so nobody plans around a guarantee that cannot exist.

How fresh it is¶

Tags are read on each run of a suite against that table and cached on the asset. So a newly-classified column takes effect from the suite's next run — including on samples captured before it was tagged, because a classification is a statement about the data rather than about the moment it was read.

If the tags cannot be read — no permission on the tag, a warehouse that is down — DataQ logs it and behaves exactly as it did before the tag existed. It never infers a clearance from a failed lookup, because a wrong guess in that direction un-masks data.

Fail-closed mode¶

For a dataset where "unclassified" should mean "do not show", set require_classification on the suite's column policy. Nothing row-level is then surfaced unless a column is explicitly cleared — by a public-family tag or by your own identifier_column. The classifier is not consulted at all, which is the point: a column called field_7 full of national ID numbers looks harmless to a name heuristic.

It is off by default, and deliberately: a fully-masked failing row is unactionable, so this is a trade you make for a regulated dataset rather than one made for you.

Data residency¶

Where data lives, and what can take it elsewhere. GDPR Ch. V asks a controller to answer both; this section describes how DataQ answers them, and GET /api/v1/admin/deployment gives the answer for your instance, read live from the running system (workspace-admin only) so it can be checked rather than trusted.

One declared jurisdiction per deployment. The unit that matters is the jurisdiction — the country whose law applies — because that is what GDPR Ch. V keys on; a cloud region is how a provider spells a location within one.

The primary region is set by a single IaC variable (azure_location / aws_region) and declared to the app as DEPLOYMENT_REGION. The app's value is a declaration, not a verification — software cannot confirm which datacentre its database sits in — and it is reported as null when unset, so an auditor sees a gap rather than a guess.

This page deliberately does not state where any particular deployment sits. DataQ is customer-deployed: your regions are yours, they are set at deploy time, and a region printed in a document is a snapshot that silently stops being true. Read the live answer from the running instance instead — GET /api/v1/admin/deployment (workspace-admin only) reports the declared region and the enumerated transfer vectors from the system itself. A control you can query beats a paragraph you have to trust.

Where each resource sits¶

By default every resource is in the declared region, so the interesting rows are the ones that cannot be:

Resource	Region	Holds customer data?
PostgreSQL, secret store, compute, object storage	the declared region	Postgres holds the data — suites, checks, results, and the incidental personal data in failing-row samples. The secret store holds warehouse credentials; compute holds data in transit only
Telemetry (App Insights / CloudWatch / OTLP)	the sink's region — operator-chosen, and may differ	Operational metadata; PII redacted at the logger
CloudFront (AWS only)	global edge	No. Only fingerprinted static assets are cached (`/assets/*.<ext>`); every other path is pass-through, so no API response and no failing-row sample is stored at an edge location
WAFv2 Web ACL (AWS only)	`us-east-1`, unavoidably	No — the ACL is rule configuration. CloudFront-scoped ACLs exist only in `us-east-1` regardless of where the stack lives

The CloudFront and WAF rows are stated rather than omitted so that a reviewer who finds a us-east-1 provider alias in an eu-west stack finds it explained here rather than having to work out whether it matters. Both are properties of the product, not of any one deployment.

When a resource legitimately sits elsewhere¶

A deployment may attach the app to a pre-existing database server or Container Apps environment — a subscription quota, or an organisation's shared-platform policy, are the usual reasons. Its region was then fixed by whoever created it and is not something this stack chooses.

The two are treated differently, on purpose. A shared database in another region is an accepted exception rather than drift: declare it with shared_pg_expected_location, which turns "this server is somewhere else" from an unexplained mismatch into a recorded decision.

A shared Container Apps environment has no such escape hatch, deliberately — it is compared against azure_location and a mismatch fails the plan outright. The asymmetry follows from blast radius: one out-of-region database is a single resource you can reason about, whereas the environment's region is inherited by every Container App and Job in the stack, so accepting a mismatch there silently relocates the entire deployment. Moving the app's compute to another jurisdiction should require changing the declaration, not setting an override.

The basis on which such an exception is acceptable is the part worth checking, and it is jurisdiction, not region. Two regions inside one country engage no Ch. V transfer. Two regions straddling an adequacy boundary do, whatever the provider's console calls them — so an operator in a regime that cares about sub-national placement, or one deploying across an EU boundary, must consolidate rather than inherit somebody else's reasoning.

The assertions that catch drift¶

The Azure stack can share a Container Apps environment, declared as a data source — so its region is set by whoever created it, and every Container App and Job in the stack inherits it (a Job must sit in its environment's region). Left unchecked, moving or recreating that shared environment elsewhere would relocate all of the app's compute with a clean apply and no signal.

aca.tf therefore carries a postcondition comparing the shared environment's actual location against var.azure_location, so a mismatch fails the plan with a message naming both. For a Ch. V control, "we did not notice the jurisdiction changed" is the whole failure mode.

A shared database gets the same comparison as a check block, which warns rather than blocking. A postcondition there would fail every apply until someone migrated a database, and the right response to "the DB moved" is a decision, not a rollback.

It compares against shared_pg_expected_location, which defaults to azure_location — so for an ordinary single-region deployment the two are the same thing and the check is silent. Setting the variable is how a deployment records an accepted exception. That indirection is the difference between a check that still works and one that warns on every plan forever: an accepted exception must not cost you the detector, and it must not hand the noise to every other deployment either.

One drafting lesson, kept because it argues for the paragraph above. An earlier version of this page tabulated a specific deployment's regions. One row asserted the database sat in the declared region; checking the running deployment showed it did not, and had not for some time. Nobody had lied — the document had simply been written once and the infrastructure had moved. That is the failure mode of publishing per-deployment facts, and it is why this page now describes the mechanism and points you at the live endpoint for the values.

What can move data out¶

Enumerated at GET /api/v1/admin/deployment — enumerated rather than derived, so a vector that is switched off still appears and an auditor can see it was considered:

Alert delivery — webhooks and email carry check names, statuses and, when a failing sample is included, redacted sample values. The destination is operator-configured and its location is outside DataQ's knowledge.
Telemetry — traces and logs to an operator-chosen sink; PII is redacted at the logger, so this is operational metadata rather than warehouse values.
MCP AI clients — live today. /mcp serves run results, redacted failing samples and check configuration to whatever AI client holds a valid PAT. The model provider behind that client, and its jurisdiction, are chosen by the token holder and are outside DataQ's knowledge. This is the more consequential of the two LLM entries, and an earlier draft listed only the other one.
LLM intelligence — the outbound direction, DataQ calling a model on its own behalf: built, off by default until an admin configures a provider and credential, sent to that admin-chosen endpoint. What each feature sends differs and is enumerated exactly, not summarized as one shape:
- SQL generation — the target's column names, plus each column's null rate and distinct count if the caller asks for the profile. No min/max, no frequent values, no cell content.
- Check suggestions — column names with null rate, distinct count, min/max, and each non-sensitive column's five most frequent literal values (masked to a placeholder on any column the column-policy or warehouse-tag floor marks sensitive — an unclassified free-text column is not masked, so its most common cell contents do leave the deployment).
- Root-cause narratives — the stored evidence card (check/asset identifiers, statuses, metric values) plus the triggering check's own observed value — routed through the same column-policy/warehouse-tag floor every other results surface applies — and its expected value (a check-authored threshold, not warehouse data, so never masked). No column profile on this path.

Never raw sample rows, on any path. Every call — feature invocation or admin test probe alike — is recorded with requester and token counts. - Sign-in email — email-OTP codes to user addresses via the configured SMTP relay: account identifiers rather than warehouse content, relay operator-chosen. - Secret store — warehouse credentials in Key Vault / Secrets Manager / OpenBao. Not customer data, but a remote store is a location, and the credentials it holds unlock the systems the customer data lives in.

Deploying into another jurisdiction is a variable change, not a fork: the same images and the same IaC take a different region. What DataQ does not do is verify the result — that remains the deploying organization's attestation, which is the correct split for a customer-deployed product.

What this is: a summary of how the technical controls above map onto the major data-protection regimes. What this is not: a legal compliance certification. Much of GDPR/HIPAA is organizational (DPAs, BAAs, DPIAs, consent, lawful basis, breach process) and is the deploying organization's responsibility, not the codebase's.

Roles & deployment model¶

DataQ ships as customer-deployed, bring-your-own-license software, not multi-tenant hosted SaaS. That fixes the data-protection role split:

The deploying organization is the data controller (GDPR) / covered entity or business (HIPAA / CCPA). It chooses the region, owns the warehouse data, holds any LLM credential, and carries consent / lawful-basis / DPA / BAA obligations.
DataQ (the software) is a processor / business associate. Its job is to provide the technical controls — security of processing, minimization, deletion levers, auditability — the controller needs to be compliant.

We do not market "DataQ is GDPR/HIPAA compliant" — no software can be. The honest claim is processor-grade technical controls for GDPR / CCPA / HIPAA workloads: an access audit trail, data-subject-rights tooling, authoritative PII/PHI classification, a declared and enforced deployment region, and encryption at rest — each covered in its own section above.

Personal data / PHI appears only incidentally in DataQ — in failing-row samples and, for a set-oriented expectation's full observed-value list, in a result's observed_value. There is no core people-database; HIPAA applies only when a customer points DataQ at PHI, GDPR only to EU personal data.

Per-regulation summary¶

Regulation	Applies when	Controls in place
GDPR	EU personal data in scope	Privacy-by-design handling (logger-level redaction, retention limits); access audit trail (Authentication & access); data-subject-rights export/erasure (below); declared + enforced residency (Data residency)
CCPA / CPRA	CA residents' data, "business" threshold	No sale of data; deletion via retention purge + the data-subject-rights erasure lever below; right-to-know via the export lever
HIPAA	Customer processes PHI	Encryption in transit/at rest (Encryption); access control (roles + suite grants); audit controls — config changes and data reads recorded, admin-queryable, and hash-chained for tamper-evidence (unanchored to an external log sink by default); a BAA remains the deploying organization's obligation

Data-subject rights (export / erasure)¶

DataQ has no people-table of its own, so a subject is identified the same way the controller's own warehouse identifies them: an Admin-only (column, value) workflow. POST /api/v1/admin/data-subject-requests/export returns every matching failing-row-sample cell unredacted (the access/portability right) across every suite in the workspace, and records an access-audit event. The erasure counterpart removes only the matching row/cell — the rest of a result's captured sample survives. Both also reach the stored incident evidence snapshot, a third persisted copy of the same value that sits outside the retention clock, and report result and incident hits separately. Both run synchronously and are recorded in the audit trail. Full mechanism and a verification walkthrough are in the data-subject-rights runbook. The controller's own warehouse remains their own responsibility — this covers only what DataQ itself has captured.

Audit-log tamper-evidence¶

The REVOKE UPDATE, DELETE on audit_events (Authentication & access) guards against accidental in-app mutation only — the table's owner can grant the privileges back, and the retention sweep does exactly that, by necessity. Every event is additionally hash-chained to the one before it: GET /admin/audit-events/verify (workspace-admin gated) recomputes the chain and reports the first break, and a daily scheduled task runs the same check unattended. Retention purges checkpoint the boundary they create before deleting, so a documented purge reads as clean rather than as tampering.

The chain alone is not the tamper-evidence claim — an external anchor is. An actor with write access to the whole table could recompute the chain forward from wherever they edited, proving nothing to that specific adversary. An optional TamperAnchor publishes the chain head, HMAC-signed, to an operator-controlled endpoint outside the database (a webhook to a separate log sink or append-only object store) — DataQ never reads it back, which is the whole point. It is off by default; an operator whose regime requires a provably unaltered log must configure it.

Organizational artifacts¶

Templates and reference documents for the parts of compliance that are genuinely organizational, published alongside this page:

Sub-processor / third-party disclosure — every external service the software can send data to.
DPIA input sheet — the personal-data inventory only the software can supply: data classes, per-location retention, and controls.
Breach-notification runbook — a reference-deployment procedure plus a template for customer deployments.
DPA / BAA templates — drafted with accurate technical annexes and marked counsel-review-required; not an engineering sign-off. Consent/lawful-basis guidance stays with the deploying organization.

Reporting a vulnerability¶

Please report suspected security issues privately to the maintainers rather than opening a public issue.