[mforce]

Why

A blocking Trivy scan stalled every PR and every publish on one base-image CVE that no PR could fix (#1006, CVE-2026-84782). Keycloak, Harbor and Kyverno scan on a schedule, report to the Security tab and file issues; none block PRs on an image scan. Owner decision: no scan in CI, no ignore file, report instead.

Closes #1006

Scope

• ci.yml image job: Trivy step removed and the job renamed Image build + smoke test (<arch>). The two-architecture matrix, build, lock-drift guard, boot smoke test and verified export are untouched and still gate publish; no continue-on-error.
• image-scan.yml (new, weekly Mon 07:00 UTC plus workflow_dispatch): scans the published :sha-<commit> of the newest of main's last 30 commits that has one, on linux/amd64 and linux/arm64, each by its own manifest digest resolved from the index. Trivy is pinned (setup-trivy version: v0.75.0; Dependabot does not bump that input, so bump it by hand, recorded in Deploy: container/image hardening — non-root USER, pinned base digests, image vulnerability scan #267). SARIF goes to the Security tab with category: image-scan-<arch>. Three jobs, each with its own permissions: resolve and scan read only (scan adds security-events: write), and only issues holds issues: write, so the third-party Trivy action never runs beside it.
• .github/scripts/image-scan-issues.mjs (+ test, run in the classifier self-test job): one issue per vulnerability id (open or closed), matched by the id in the scanner's own title ([HIGH] CVE-… in <pkgs> (container image)), never by body. New issues get dependencies and docker, no priority label. A found id gets a comment; a cleared id gets a comment naming the commit and both digests, then is closed. It closes only issues in the scanner's title shape carrying both labels and authored by github-actions (a human-filed issue is never closed; updates stay title-only), and only when both architecture reports parse and name the image; otherwise it exits 1 before touching any issue. The issues job also needs both scan jobs to succeed.
• Docs found by git grep -ni trivy: AGENTS.md (Deploy: container/image hardening — non-root USER, pinned base digests, image vulnerability scan #267, CI: skip the image and web jobs on pull requests that change no code #782 paragraph, CI builds the runtime image but never publishes it — deploy-by-digest has no artifact to point at #351 publish.needs, Publish multi-arch images so Cluckwork runs on arm64 (Raspberry Pi and ARM VPS) #995), Deploy: container/image hardening — non-root USER, pinned base digests, image vulnerability scan #267 record (reversal, reason, accepted costs), CI: dependency-vulnerability and SAST gates #146, Runtime image lacks libgssapi_krb5.so.2 — Npgsql logs a load error on every connection #332, CI builds the runtime image but never publishes it — deploy-by-digest has no artifact to point at #351, CI: skip the image and web jobs on pull requests that change no code #782, Publish multi-arch images so Cluckwork runs on arm64 (Raspberry Pi and ARM VPS) #995 records, docs/releasing.md, SECURITY.md, deploy/README.md. The CI: dependency-vulnerability and SAST gates #146 NuGet and npm gates are untouched. Timings in the CI: measure where Build and test's 10 minutes go, and what the backend suite covers #775 and CI: skip the image and web jobs on pull requests that change no code #782 records name the old check; they are dated measurements and are left as written.

dependabot.yml and changed-paths.mjs changed by comment text only: each named the Trivy gate as a behaviour. No logic changed; the #782 classifier tests still pass (25/25 with the new ones).

Tradeoffs

• Accepted cost (recorded in Deploy: container/image hardening — non-root USER, pinned base digests, image vulnerability scan #267): published and released images are no longer scanned before they ship. A new vulnerability can go unreported for up to a week, and a release can ship an unscanned image. The digest artifact now proves built and boot-tested, not scanned. workflow_dispatch on image-scan.yml scans on demand (dry_run available); not a required release step.
• If an owner closes a scanner issue while the CVE persists, the next scan files a new one (open issues only, as specified).
• Fixes arrive as Dependabot docker bumps; the next scan closes the issue.

Blast Radius

CI and docs only. publish and promotion are unchanged apart from no longer waiting on a scan. image-scan.yml cannot block anything. Left unmerged, every code PR and every main push stays red or unpublished while the CVE stands.

Verification

Dry-run against the live repo, using real Trivy reports of the published sha-ad258d8f image (both architectures). #1006 is not matched, and nothing was created, edited or closed:

# (a) live dry run 1 (closed #1006 has no CVE in its title and does not match)
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would create a new issue for CVE-2026-84782: [HIGH] CVE-2026-84782 in libssl3t64, openssl (container image)
exit 0
# (a) live dry run 2 (closed #1006 has no CVE in its title and does not match)
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would create a new issue for CVE-2026-84782: [HIGH] CVE-2026-84782 in libssl3t64, openssl (container image)
exit 0
# (b) open scanner issue, CVE present -> update
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would update #9001 for CVE-2026-84782 with a comment
exit 0
# (c) successful clean scan: bot issue #9002 closed; human-authored #9003 and human issue #9004 untouched
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would close #9002 (CVE-2026-99999 no longer found)
exit 0
# (d) arm64 scan failed -> nothing changes
image-scan-issues: arm64 report is not JSON: Unexpected token 'E', "Error: pul"... is not valid JSON
exit 1
# (e) closed issue, CVE present -> add ignored
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would add the ignored label and a comment to closed #9005 (CVE-2026-84782 still present, ignored)
exit 0
# (e, second run) same issue now labelled -> nothing changes
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
skip CVE-2026-84782: #9005 is closed and already ignored
exit 0
# (f) reopened, labelled ignored, present -> remove label, then update
image-scan: <image> scanned on amd64 and arm64; 2 action(s) (dry run)
would remove the ignored label from #9006 (CVE-2026-84782)
would update #9006 for CVE-2026-84782 with a comment
exit 0
# (g) closed + ignored, CVE absent -> remove label
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would remove the ignored label from #9007 (CVE-2026-99999)
exit 0
# (h) open and closed both match -> update the open one only
image-scan: <image> scanned on amd64 and arm64; 1 action(s) (dry run)
would update #9009 for CVE-2026-84782 with a comment
exit 0

Reports are of the published sha-ad258d8f image, scanned by child digest with Trivy 0.75.0. (b) to (h) feed the decision logic fixture issues (--issues, bot-authored unless noted); the tests cover each case and each was proven red by reverting its logic (7 mutations); the unit tests cover the same plus "human issue never closed" and "id in another issue's text not matched". Mutation checks: dropping the label, author or architecture check, or escaping pipes before backslashes, each turned a test red. actionlint shows only the 4 existing infos in release-please.yml; the two ImagePin tests and SchemaDocsTests/SolutionTestProjectSplitTests pass.

The scan step itself ran as the pinned Trivy 0.75.0 container locally (same flags). Its first run in Actions is untested: after merge, dispatch it once with gh workflow run image-scan.yml --ref main -f dry_run=true and check that (1) resolve picks a commit, (2) both scan legs go green with a table in the summary and a SARIF category each under Security > Code scanning, and (3) the issues summary says it would create an issue for CVE-2026-84782. Then dispatch with dry_run=false to file it, and check that the issue it creates is authored by github-actions. The bot's author login is inferred, not observed: gh reports GITHUB_TOKEN-filed issues as app/github-actions and the code normalises that, but no issue in this repo has ever been filed by it. If the author differs, closing silently never happens.

This PR's own CI should show the image job green on both architectures with no scan.

Review round 1 (a4bd642d)

Pinned Trivy; per-architecture manifest digests (RepoDigests was the index digest for both); author check for closing; honest close comment ("no longer reported as a fixable HIGH or CRITICAL"); picker fails on anything but not-found; backslash and @ escaping with tests; persist-credentials: false on the issues checkout; Dockerfile comment reworded.

Round 2: resolve pins the index digest once and both scan legs take their child manifest from it; a test now asserts the advisory autolink (breaking the link logic turns it red).

Ignore by closing (owner rule)

Closing a scanner issue ignores that CVE; reopening un-ignores it. Matching is by the CVE id in the scanner's title shape, open or closed, never by body, so closed #1006 (no CVE in its title) does not match.

• Open match: update it (open wins over closed); remove ignored first if it carries the label.
• Closed match only: no new issue and no reopen. Add ignored once with one comment in the same step (the comment, then the label); a run that finds the label already there posts nothing, and removing the label never posts. A CVE that cleared (label removed) and later returns is re-labelled with a new comment.
• No match: create.
• CVE absent from a successful scan: close an open scanner-authored issue (and drop ignored if present); remove ignored from a closed issue, since it means "still present".
• A failed or partial scan changes no issue and no label. The ignored label is created idempotently by the issues job (gh label create --force); issue listing now paginates over GET /issues?state=all.

Accepted costs (recorded in #267): an ignore has no expiry, and a CVE that returns after being fixed (its issue closed by the scan) is not reopened and gets no new issue; it is re-labelled with a comment, which is the only notification.

The comment posted when ignored is added (rendered from the real reports):

Weekly image scan: CVE-2026-84782 (HIGH) is still present as of `sha-ad258d8f21a517bd32ecd214526318f511234abd` on linux/amd64, linux/arm64:
- linux/amd64: `sha256:f932eda3488acdb96319153ac4ff745b414671e1f4703f43bd5dc5b202ad709e`
- linux/arm64: `sha256:6945ed042ae2ed31749ea8a0b42b83fd48a7b12f866395c7493e410e2c93d03e`

This issue is closed, so the scan treats CVE-2026-84782 as ignored (label `ignored`) and files no new issue. Reopen this issue to stop ignoring it.

Scan run: https://github.com/mforce/cluckwork/actions/runs/1

Tests: (e2) label plus one comment, second run no comment; (e3) already labelled, no comment; (g2) label removed on clear, CVE returns, label and comment again; no comment on unlabel; and a live-run test with a stub gh showing the order label create, issue comment, issue edit, then nothing on the second run. Each was proven red by reverting its logic.

[mforce]

Review record — Claude Opus 5.5 (local agent, read-only), 3 rounds

Reviewed at the owner's request. CodeRabbit was not triggered.

Round 1 — 32a94ede: 1 must-fix, 2 should-fix

• Must-fix — Trivy itself was unpinned. aquasecurity/setup-trivy was pinned by SHA, but its version input defaults to latest. That was checked in the action's action.yaml at the pinned SHA. So every weekly run would have installed the newest Trivy release: the same exposure as the 2026-03 trivy-action compromise, in a job with security-events: write and a GHCR login. It was also a regression, since the removed CI step pinned a version. Fixed: version: v0.75.0. Dependabot doesn't manage that input, so it is bumped by hand, and the Deploy: container/image hardening — non-root USER, pinned base digests, image vulnerability scan #267 record says so.
• Should-fix: CodeQL raised a new HIGH alert because cell() escaped | but not \. Fixed, with a test.
• Should-fix: the Dockerfile header still said CI scanned the image. It never names Trivy, so a grep for Trivy missed it. Fixed.
• Hardening applied:
  • Closing requires the bot to be the issue's author, so a human-filed issue in the scanner's title format is never closed.
  • The close comment now says "no longer reported as a fixable HIGH or CRITICAL" rather than "no longer found".
  • The commit picker fails on any error other than "not found", instead of silently scanning an older image.
  • @ from vulnerability-database text is neutralised.
  • persist-credentials: false.
  • Per-architecture digests are named.

Round 2 — f33fe711: clean, 1 test gap

• The "not found" detection was probed live: buildx v0.37.1 prints not found and exits 1.
• The author check was mutation-checked: a wrong login fails 4 tests, and a bot-filed issue does close.
• Test gap: breaking the advisory-link logic left every test green. Fixed: an assertion, proven to fail when the logic is broken.
• The index digest is now resolved once and shared by both architecture legs, so they can't scan different indexes.

Owner-requested additions

• Closing an issue ignores its CVE. Matching covers open and closed issues, by the CVE id in the scanner's title format, never by body, and an open issue beats a closed one.
• A closed match is never reopened and no duplicate is filed. It gets an ignored label and one comment, only when the label is added — never weekly, and never when it's removed.
• Reopening un-ignores the CVE.
• A cleared CVE closes an open issue, or removes ignored from a closed one. A CVE that returns is re-labelled with a comment.
• The owner chose that any closed issue counts as ignored, including ones the scanner itself closed as fixed.

Round 3 — e6422750: clean

• Pagination verified live: 446 issues, matching the search API total exactly, with every pull request excluded.
• No existing issue matches the scanner's title format, ci: move image vulnerability scanning out of CI to a weekly scan of published images #1006 included, so nothing will be ignored or touched by mistake.
• Every combination of open/closed, labelled/unlabelled and present/absent was walked. Each issue gets at most one action per run, and a partial failure exits non-zero and converges on the next run.
• Six mutation claims were spot-checked by reverting each branch, and every revert turned tests red.

Optional follow-ups, not applied

• The index lookup runs after docker logout, so it relies on the package being public. If it went private, the step would fail with a clear error, not silently.
• gh label create --force resets the ignored label's colour and description whenever it adds the label.

CI on e6422750

18 passed, publish skipped (PR only). Both Image build + smoke test legs pass with the scan removed — on today's main they cannot. CodeQL is green.

After merge

1. main publishes again — the first image since ad258d8f. Release chore(main): release 0.1.5 #993 can then be promoted.
2. Dispatch the weekly scan workflow once. It should open a new issue for CVE-2026-84782, authored by the bot. That author login is inferred, not yet observed, and closing depends on it, so check it.

The review loop was stopped deliberately after round 3, which found no defects.