Argo Rollouts¶
An Argo CD PostSync gate reports a bad version after it is live:
a failed gate fails the sync and leaves the new version serving. A canary
analysis is the blocking path. The new ReplicaSet gets a share, veridor run
probes it through the canary Service before it is promoted, and a failed run
aborts the rollout and keeps the stable ReplicaSet. Of the Kubernetes
packagings, this is the only one that keeps a failing version from taking all
the traffic.
The Veridor repository ships:
deploy/argo-rollouts/veridor-analysistemplate.yaml: anAnalysisTemplatenamedveridor-gatewith one metric using the job provider.deploy/argo-rollouts/rollout-example.yaml: a canary Rollout withcanaryService/stableServiceand the stepssetWeight: 50,analysis,setWeight: 100.
Not published
The template's default image, ghcr.io/veridor-authz/veridor:0.1.0, does
not exist. Build the image from the repository's Dockerfile and pass it as the
image argument.
The AnalysisTemplate¶
One metric, count: 1, failureLimit: 0, job provider. The Job is the same
restricted-profile pod the veridor-gate chart
renders (runAsNonRoot, uid 10001, seccompProfile: RuntimeDefault, no
privilege escalation, read-only root filesystem, all capabilities dropped, no
service-account token), with backoffLimit: 0 and activeDeadlineSeconds:
600. It runs:
veridor run --config /config/veridor.json --output-dir /work/.veridor \
--base-url {{args.base-url}} --fail-on {{args.fail-on}} --require-complete \
--wait-for-target {{args.wait-for-target}}
| Argument | Default | What it is |
|---|---|---|
base-url |
none, required | The canary Service. A gate pointed at the stable Service measures the version already live. |
config-map |
veridor-config |
ConfigMap holding veridor.json. |
secret |
veridor-credentials |
Secret imported with envFrom; its keys must be the variable names the configuration references (VERIDOR_OWNER_TOKEN). |
image |
ghcr.io/veridor-authz/veridor:0.1.0 |
The gate image. A placeholder. |
fail-on |
new+confirmed |
--fail-on. |
wait-for-target |
"60" |
--wait-for-target: seconds to wait, before the first request, for the canary Service to accept a TCP connection. The analysis can start as the canary pod turns Ready, before the Service routes to it; probes sent then are refused, and exit 3 aborts the rollout. It never changes a verdict. "0" turns it off. |
The Job's other arguments are fixed in the template; edit it to add Veridor
flags. The secretRef is optional: true: a missing Secret leaves the
variables unset and Veridor refuses at once (exit 2, naming the variable),
which fails the measurement instead of the pod waiting for its deadline.
A canary example¶
kubectl -n shop create configmap veridor-config --from-file=veridor.json
kubectl -n shop create secret generic veridor-credentials \
--from-literal=VERIDOR_OWNER_TOKEN=... --from-literal=VERIDOR_OUTSIDER_TOKEN=...
kubectl -n shop apply -f veridor-analysistemplate.yaml
strategy:
canary:
canaryService: invoices-api-canary
stableService: invoices-api-stable
steps:
- setWeight: 50
- analysis:
templates:
- templateName: veridor-gate
args:
- name: base-url
value: http://invoices-api-canary:8080
- setWeight: 100
With no trafficRouting, Argo Rollouts splits by replica count and adds the
pod-template-hash to each Service's selector, so the gate reaches the canary
through invoices-api-canary while clients of invoices-api-stable stay on the
stable ReplicaSet. The e2e measured the pinned selector on
invoices-api-stable, and one ready canary pod and one ready stable pod when
the analysis started. That pinning is also why the gate here probes the new
version and not the one being replaced, which a PostSync gate on a
RollingUpdate app does not guarantee
(Kubernetes and Argo CD). With a
traffic router (Istio, NGINX, ...) the same steps shift traffic by weight;
that setup was not measured.
The wait-for-target wait holds the first request until the canary Service
accepts a TCP connection. An accepted connection means a port is open, not
that the application behind it is ready.
Abort semantics: every non-zero exit aborts¶
The job provider has two outcomes, not three. A Job that completes is a
Successful measurement; a Job that fails is a Failed one, and with
failureLimit: 0 one failure aborts the rollout.
| Veridor exit | Job | Measurement | Rollout |
|---|---|---|---|
0 |
Complete | Successful | continues to setWeight: 100, promoted |
1 findings |
Failed | Failed | aborted (status.abort: true), status.phase: Degraded, stable ReplicaSet kept |
2 refused |
Failed | Failed | aborted |
3 something went unmeasured |
Failed | Failed | aborted |
70 internal error |
Failed | Failed | aborted |
143 SIGTERM (for example the Job deadline) |
Failed | Failed | aborted |
There is no Inconclusive outcome in the job provider, so exit 3 aborts the
rollout exactly as a finding does, and so does a configuration mistake. That is
the fail-closed direction, and it means a canary that could not be measured is
not promoted. The only way to let a partially measured canary through is
Veridor's own flags: the template passes --require-complete, and removing it
lets a run with some unanswered probes pass. A run that answered nothing still
exits 3 without it.
Measured by scripts/e2e_kubernetes.sh (Argo Rollouts v1.10.0, kind), as
recorded in the Veridor repository's deploy/argo-rollouts/README.md, with this
template as-is and the example Rollout's pod replaced by the demo API; not
re-run for this page. The same script ran in a kind cluster in the project's
CI on GitHub (workflow run 42508338432, success; the repository is private):
| Step | AnalysisRun | Gate exit | Rollout |
|---|---|---|---|
| revision 1, fixed (first rollout; steps are skipped) | none | — | Healthy |
| revision 2, vulnerable canary | Failed |
1 |
Degraded, status.abort: true, stable ReplicaSet unchanged |
| revision 3, fixed canary | Successful |
0 |
Healthy, promoted |
Exits 2, 3, 70 and 143 were not produced inside a Rollout; their rows
above follow from the job provider's two outcomes and the Job failing on any
non-zero exit.
Returning to exactly the stable spec is treated by Argo Rollouts as going back to stable, skipping the steps (its documented behaviour; not measured), so it does not exercise the gate.
Related: Kubernetes and Argo CD, exit codes, install.