What a Veridor finding does and does not mean¶

Read this before you read a report. It is the tool's defining limit, and every number on this site is bounded by it.

Your configuration is the policy¶

Veridor has no independent notion of who should be allowed to see what. It reads the expectations you declared, issues the requests, and reports where observed behaviour and your declaration disagree.

So a finding means exactly one thing:

Observed behaviour differed from the expectation you declared.

And it does not mean:

The declaration was correct.

Veridor cannot tell those apart. A wrong declaration and a real vulnerability produce the same finding, with the same verdict, at the same evidence grade. If you declare that an administrator must receive 404 on a resource that administrator is genuinely entitled to read, Veridor reports a failure — a true one against your declaration, and a false one against reality.

Veridor consumes a policy statement. It cannot audit one. There is no version of this tool that could: auditing the statement would require an independent source of truth about your entitlements, and if you had one you would not need this.

Triage a finding by checking your declaration first

It is the cheaper of the two hypotheses to eliminate. Reading one line of JSON costs less than reading an authorization layer.

The oracle proves data flow, never policy¶

A marker is a high-entropy string that exists only in one identity's data. When it turns up in another identity's response, that is a measurement, not an inference: bytes that were in A's record were served to B. Veridor states it flatly, because it observed it.

What the marker cannot tell you is whether B was allowed to see them. Marker presence is a fact about data flow. Entitlement is a fact about policy, and policy comes from your config file.

Take the quickstart's confirmed finding: other_tenant received invoice_owner's marker, and you declared it should not. Now suppose your product means that identity to read that invoice, as a support role might. The run reports exactly the same evidence_grade: confirmed finding, because the marker really is in the response. Nothing is wrong with the target. The declaration is wrong, and the marker oracle cannot rescue it, because there is nothing about the data flow to rescue. DeclarationScopeTest in tests/test_marker.py builds this case.

The confirmed grade also rests on a premise you supply: that the marker is unique to that identity's data. You supply it by planting the marker. Where the run has positive evidence against that premise — a credential Veridor fabricated got the same string back — the grade caps at suspected and the data-flow sentence is not emitted at all. See verdicts and evidence grades.

Every run therefore prints the same standing sentence next to its evidence counts, the run document carries it under the interpretation key, and SARIF carries it in the tool's fullDescription and in VERIDOR-AUTHZ-001's (not yet in the other rules'), so a downstream consumer inherits the caveat instead of re-deriving it:

Veridor evaluates the expectations you declared. A finding means observed behaviour differed from your declaration; it does not establish that the declaration is correct.

The three measurements, and what each does not claim¶

Measurement	Claims	Never claims
Boundary coverage (`map`, `coverage`)	A cross-identity denial is asserted for this operation — a fact about your configuration	That the boundary is enforced, or that the assertion could ever fail
Mutation score (`mutate`)	Perturbing your declaration changes the verdict, so the check would catch a regression	That the boundary is enforced, or anything about operations your suite does not cover
A finding (`run`)	Observed behaviour differed from your declaration on this request	That the declaration was right
A candidate (`map`)	Two observations are inconsistent — a hypothesis to declare and test	That anything is wrong; it is not a finding and produces no SARIF result
Blast radius (`map`)	How many operations an identity was observed to reach, as a lower bound	How many objects those operations expose

Coverage is asserted, never enforced¶

veridor coverage opens no socket. It can score an API that was never contacted, because what it measures is whether your configuration asserts a cross-identity denial for each operation in the specification. The badge reads authz denial asserted, not covered, for exactly that reason.

An expectation that accepts both success and refusal asserts nothing and is reported as a diagnostic rather than counted. An expectation of 401 alone is an authentication assertion and never counts as authorization coverage.

A mutation score is about your checks, not your target¶

mutate perturbs your declaration and re-runs the suite. A mutant that dies tells you the check discriminates: change the declared boundary and the verdict changes with it. A mutant that survives tells you the check may be vacuous.

Neither says anything about whether your target enforces anything. A surviving mutant is not a defect in your API. And the score is only as wide as the mutants it could evaluate — a probe lost to a rate limit is now recorded errored and exits 2, rather than being dropped from the denominator, because a third of the population disappearing while the number stayed at 1.00 is the same overclaim in a quieter register.

The two scores multiply rather than add¶

Full coverage made of checks that cannot fail is worth nothing. A perfect mutation score over one operation says nothing about the operations coverage reports as untested. Neither number rescues the other, and the product — not the sum — is what your suite is worth.

The measured limits, with their bounds¶

These numbers were produced by scripts/noise_floor_study.py against scripts/noisy_server.py: an API whose authorization boundary never changes and whose everything else moves like a real one — fresh request ids and timestamps, moving ETags, a collection sorted by recency so records change page, transient 429s and 503s, and latency past the client timeout. Every change reported against it is a false positive by construction.

At the script's default noise — a 429 on 6% of requests, a 5xx on 3%, and latency past the client timeout on 5% — over seeds 1 to 9, ten runs each.

Drift false gates¶

Over ninety runs, --fail-on-drift any fired 0 times, and --fail-on-drift opened fired 0 times.

0 of 90 is not 0%. Ninety runs that did not fire bound the false-gate rate below roughly 4% with 95% confidence — Wilson interval [0.0%, 4.1%]. They cannot establish that it is zero, and no number of clean runs could. All ninety are one synthetic server at one noise profile. A real API with a different failure shape is not covered by this measurement.

Detection¶

Over the same seeds, a planted boundary opening on one fixture was caught on 87 of 90 runs — 96.7%, 95% CI [90.7%, 98.9%].

That is a per-run reliability figure for one regression on one target, conditional on a baseline that measured the cell. It is not a detection rate for authorization bugs in general, and nothing here measures that.

Where the baseline is blind, detection is 0%

This is not variance around a mean. It is one binary fact about each cell: detection is high for every cell the baseline measured, and 0% for any cell the baseline itself could not measure. A baseline recorded through one bad minute is blind there forever, because every later comparison is unknown → something, which is never "more access". Re-running changes nothing, because nothing failed.

Veridor cannot repair that, so it refuses to hide it. map warns when the matrix it has written would make a poor baseline, and any drift report against a blind baseline says so above its own counts.

These two numbers are deliberately stated apart. Adjacent rows invite the summary "0% false alarms, 97% detection", which neither number supports — one is a bound on a rate over ninety runs against a server that never changed, the other is a per-run figure for a single planted regression.

The run gate¶

The same study against veridor run — the verb that emits findings and SARIF — produced 0 of 10 runs failing the build on transients and 0 of 10 false vulnerable claims into SARIF, per seed. 0 of 10 bounds the rate below roughly 28% with 95% confidence. Ten runs is a small sample, stated as a count rather than as a rate so it is not read as a guarantee. Real leaks were caught on 62–100% of runs per seed, every one graded confirmed.

Drift is blind to anything that did not change. An endpoint that should be admin-only and has been open to everyone since before your first run never moves, so it never appears. "Nothing moved" is reported as exactly that, never as a clean bill of health.
Drift is blind to whatever map did not probe. The mutating half of a CRUD inventory is skipped by default, so those cells are unknown in the baseline and in every later run. On the demo specification that is 8 of 20 cells: drift speaks about 60% of the grid, and says so above its own counts.
run is blind to every boundary you did not declare. An unstated boundary is an untested one. That is what map and coverage are for.
A pass is not enforcement. A matched probe means the target agreed with your declaration for that one request. An expectation that accepts both access and denial matches trivially, and an exclusion licensed by an identity marker_probe was never demonstrated at the path under test.
Blast radius counts operations, not objects. It is a lower bound on reach, and it says nothing about how many records sit behind each endpoint.

Non-goals¶

None of these are on a roadmap. They are stated plainly because a security tool that overstates itself is worse than no tool at all.

No API discovery. Veridor never crawls, spiders, or guesses routes. It sends the requests you wrote, plus the controls listed below; the optional route inventory comes from an OpenAPI file you point it at.
No policy invention. Veridor has no idea who should be allowed to do what.
No fuzzing and no payloads. run sends each check once per identity, as configured, with Accept-Encoding, Cache-Control and Pragma forced. On top of those it sends each marker_probe and, unless you pass --no-prove-liveness, one control carrying a credential it fabricated per GET, HEAD or OPTIONS request whose declaration accepts only denials, excludes an owner's marker, or must prove a credential live. That credential authenticates to nothing, but a target's lockout or WAF rules can still count it. No mutation, no injection, no traversal strings.
No injection classes. SQL injection, command injection, SSRF, XSS, path traversal and their relatives are somebody else's tool.
No authentication-strength testing. Veridor does not crack tokens, analyse JWTs, test session fixation, or evaluate password policy. It presents the credentials you give it.
No proof of security. A passing run means nothing the gate fails on was found on this run: a finding the policy does not gate on, and an unanswered probe without --require-complete, exit 0 too. An expectation that held is evidence, not a guarantee.

The design commitments behind all of this¶

Changing one of these is a breaking change.

Standard library only at runtime. No runtime dependency to install or audit; building from source still fetches the build backend.
Response bodies are never persisted. Scanned in memory, discarded.
Finding fingerprints are stable. New concepts ride as properties, so an upgrade never reports your whole suite as new.
The measurement fails closed. A weak marker, an unverifiable oracle, an unestablished session, or an undefined coverage ratio is reported as such. Veridor would rather say "unknown" than say something reassuring and false.
The configuration is not a programming language. No conditionals, loops, arithmetic, or expressions. A security tool nobody can review is worthless.
Credentials are not written down. Cookie values, captured tokens and login secrets exist only in memory for the duration of a run, and no cookie jar is ever persisted, under any flag. Every value expanded from a ${NAME} is written and printed as ${NAME}; the stated limits (a value under 8 characters, a literal you typed yourself) are on safety posture.