coverage — what your policy asserts¶

veridor coverage compares two files: your configuration and an OpenAPI specification. For each operation in the specification it reports whether some check asserts a cross-identity denial on it. It issues no request and opens no socket, so it runs in an air-gapped job and against a specification for an environment the runner cannot reach.

The number is a fact about what you wrote down. It says a denial is asserted for an operation. It never says the denial is enforced; only veridor run measures that, by sending the requests. See coverage is asserted, never enforced.

When to use it¶

To find the operations your configuration says nothing about. The uncovered list is the useful part of the report.
As a CI step that needs no target and no credentials, or to publish a badge.
After veridor init, to see which scaffolded checks still credit nothing.

The same measurement runs inside veridor run when you pass --openapi, and inside veridor map (see boundary coverage). The standalone command gives it without sending anything.

Flags¶

Flag	Default	Does
`--config PATH`	required	the Veridor configuration
`--openapi PATH`	required	an OpenAPI 3 or Swagger 2 specification. JSON only
`--coverage PATH`	none	also write the report as JSON (`"kind": "veridor.coverage"`)
`--badge PATH`	none	also write a shields.io endpoint JSON file for a badge
`--base-path PREFIX`	auto-detect	the prefix to put in front of every specification path
`--allow-no-matches`	off	when no check path matches any operation, report coverage as not measured and exit `0`, instead of exit `2`. A specification with no in-scope operation still exits `2`

run takes the same --openapi, --coverage, --badge, --base-path and --allow-no-matches flags. --coverage and --badge may not name the configuration or the specification (--coverage must not overwrite a configuration, baseline, or specification, exit 2).

Run it¶

veridor coverage --config fixtures/veridor.demo.json --openapi fixtures/openapi.demo.json --coverage coverage.json --badge badge.json

Veridor boundary coverage
  spec:      openapi.demo.json (OpenAPI 3.0.3, 'Veridor Demo API')
  config:    veridor.demo.json
  base path: (none)  (no prefix; 2 of 2 check paths matched)

Authorization boundary coverage: 2/5 operations (40.0%)

  Coverage measures whether a cross-identity denial is ASSERTED for each
  operation. It does not measure whether authorization is ENFORCED.
  Run `veridor run` to test enforcement.
…
Corroborated: 2/5 operations
…
3 operations have no cross-identity denial asserted:
  GET    /api/invoices
  DELETE /api/invoices/{id}
  POST   /api/admin/audit/export

Path-level coverage (secondary): 2/4 paths (50.0%)

Exit code 0. The demo configuration declares two checks, so two of the five operations are covered.

What is counted¶

The unit is an operation: one method on one path. GET, POST, PUT, PATCH and DELETE are counted. Other methods, and paths the parser cannot use, are listed as excluded rather than counted.

Each identity's status list on each check is classified:

Class	When	Counts
`denial_only`	every status is a refusal, and at least one is an authorization refusal (`403` or `404` by default)	yes
`authn_only`	every status is `401` or `407`	no: an authentication assertion
`allow_only`	every status is `2xx`	no, but it can corroborate
`permissive`	a `2xx` mixed with a refusal	no: it accepts both outcomes and asserts nothing
`redirect`	a `3xx`	no
`server_error`	a `5xx`	no
`ambiguous`	any other status, or no status	no

An operation is covered when at least one check that matches it has a denial_only identity. It is also corroborated when the same check has an allow_only identity: the denial cannot then pass only because the route is absent. A covered operation that is not corroborated is counted under weak_coverage in the JSON report.

A check path is matched to a specification path after the path component of target.base_url is put in front of it, because that is the URL run sends. Paths are compared case-sensitively; a near miss is reported as a diagnostic. A check path that still contains a placeholder such as {id} credits nothing and is reported as templated_check_path. Path-level coverage, printed last, counts a path as covered when any of its operations is.

Denial statuses¶

The authorization refusals are 403 and 404. A top-level coverage block can replace them:

"coverage": {"denial_statuses": [403]}

The list may hold only 4xx statuses other than 401 and 407. With [403], the demo's other_tenant expectation of [404] no longer counts:

veridor coverage --config demo-403.json --openapi fixtures/openapi.demo.json

…
Authorization boundary coverage: 1/5 operations (20.0%)
…
Diagnostics (1):
  ambiguous_expect  expectation [404] for identity 'other_tenant' on check 'invoice-tenant-isolation' contains a status that is neither an authorization denial nor an authentication denial, so the set is not denial-only; not counted

Denial statuses in use (declared by config): [403]
…

demo-403.json is fixtures/veridor.demo.json with that block added. "denial_statuses": [401] is refused with exit 2:

veridor: boundary coverage was not measured: coverage.denial_statuses must be a non-empty list of 4xx authorization refusal statuses; 401 and 407 are authentication and never count as an authorization denial

The rules for the block are in top-level keys.

The base path¶

Specification paths are relative to the specification's servers URL; check paths are relative to target.base_url. Coverage scores each candidate prefix (the specification's server paths, the path of target.base_url, and no prefix) by how many check paths it makes match, and prints the result on the base path: line. When none fits, state it:

veridor coverage --config fixtures/veridor.demo.json --openapi unprefixed.json --base-path /api

…
  base path: /api  (from --base-path; 2 of 2 check paths matched)

Authorization boundary coverage: 2/5 operations (40.0%)

unprefixed.json is the demo specification with /api removed from every path and no servers. Without --base-path the same command exits 2 (below).

The badge¶

--badge writes a file in the shields.io endpoint badge format. From the run above:

{
  "schemaVersion": 1,
  "label": "authz denial asserted",
  "message": "40.0% asserted",
  "color": "red",
  "cacheSeconds": 300
}

The label says asserted because nothing was sent. The colour is brightgreen from 95%, green from 90%, yellow from 75%, orange from 50%, and red below. Two cases never show a percentage:

Unreviewed. While the configuration still has the _generated_by key that veridor init writes, the message is unreviewed, the colour lightgrey, and isError is true. The text report prints THIS NUMBER IS NOT YOURS YET with the reason. Delete the key once you have read the checks.
Not measured. When the measurement is undefined or refused, the message is unknown, the colour lightgrey, and isError is true. A previous green badge is overwritten, not left in place.

When the number is undefined¶

If no check path matches any operation, or the specification has no in-scope operation, coverage is undefined. It is not reported as 0%:

veridor coverage --config fixtures/veridor.demo.json --openapi other-api.json --coverage coverage.json --badge badge.json

veridor: boundary coverage was not measured: none of the 2 configured check paths matched any of the 1 operations in other-api.json. Boundary coverage is undefined and is not reported as 0%.

  base-path candidates scored (no candidate produced any match):
    (none)                   0 of 2 check paths
…
  Suggestion: the config and the spec probably describe different prefixes. Pass
  --base-path <prefix> to state the spec's prefix explicitly, or confirm the spec
  covers the environment under test. Use --allow-no-matches to accept this.
…
Authorization boundary coverage: not measured (no_matches)
…

Exit code 2. coverage.json was written with "status": "no_matches" and "coverage": null, and badge.json with "message": "unknown". With --allow-no-matches the same report prints and the exit code is 0. A specification with no in-scope operation prints not measured (empty_inventory) and exits 2 with or without the flag.

The JSON report¶

--coverage writes one document. Its top-level keys are schema_version, kind, generated_at, spec, config, base_path, claim, policy, status, coverage, summary, operations, uncovered, weak_coverage, excluded, diagnostics, spec_warnings and identity_roles. From the run above:

"status": "measured",
"coverage": 0.4,
"summary": {"operations": 5, "covered": 2, "uncovered": 3, "corroborated": 2,
            "weak": 0, "authentication_only": 0, "excluded": 0,
            "unmatched_checks": 0, "diagnostics": 0, "path_coverage": 0.5,
            "paths": 4, "paths_covered": 2}

Each entry in uncovered carries a reason: no_check, authentication_only, permissive_only, allow_only or unclassified_expectations. identity_roles labels each identity by the statuses it was declared with across all checks: challenger (denials only), principal-owner (allows only), contextual (both), authenticator (401 or 407 only), or unclassified.

When the configuration or specification does not load, or the coverage block is refused, the file is rewritten with "status": "not_measured", a null figure and a reason, so a later step never reads the previous run's figure.

Credentials are not needed¶

veridor coverage loads and validates the configuration, but an unset ${ENV_VAR} does not stop it, because nothing is sent. The init scaffold, with VERIDOR_OWNER_TOKEN and VERIDOR_OTHER_TENANT_TOKEN unset, measured 3/5 operations (60.0%) and exited 0; veridor run on the same file refused with environment variable 'VERIDOR_OWNER_TOKEN' is not set.

Exit codes¶

Code	Meaning
`0`	coverage was measured, whatever the figure; or no check path matched any operation and `--allow-no-matches` was passed
`2`	coverage is undefined, the `coverage` block was refused, the configuration has no `checks` (`checks must be a non-empty list`), an input did not load, or an output path was refused

A low figure never changes the exit code. Under veridor run --openapi, the probes run and their findings print first. An undefined figure then makes the run exit 2, unless a finding failed the gate, which keeps 1; under --fail-on none it does not move the exit code. See veridor run exit codes.

What it does not do¶

It does not send a request, read a response, or test enforcement.
It does not check that an expectation is right. A check that declares the wrong identity denied still counts.
It does not discover routes. An operation missing from the specification is missing from the denominator.
It does not read YAML. Convert the specification to JSON first.
It does not count 401 or 407 as an authorization boundary, and it does not count an expectation that accepts both success and refusal.