Only signed images run here: Kyverno, cosign and the supply-chain gate

A registry is not a trust boundary. Anyone who can push to it — a compromised CI token, a stray docker push from a laptop — can put an image in front of your cluster, and kubectl will happily run it. Signing closes that gap: the cluster only starts a container whose image carries a signature it recognises. Here's the pipeline that enforces it on every deploy of this site.

Sign at build, not by hand

CI builds every image with kaniko (no Docker daemon in the runners) and pushes it to our in-cluster registry. On a merge to main, one more step runs before the image is eligible for production:

cosign sign --yes \
  --key env://COSIGN_KEY \
  --allow-http-registry \
  --allow-insecure-registry \
  "$IMAGE_REF"

Key-based, not keyless. Keyless signing wants Fulcio for short-lived certs and Rekor for the public transparency log — great when your registry is public, awkward when it's a self-hosted service on a private cluster network. We sign with a fixed cosign key instead: the private half and its password live in CI secrets, the public half lives in the policy that verifies it. We still push the signature to the public Rekor log (the runners have egress for that one call) so there's an independent timestamped record outside our own infrastructure, even though the policy verifies the bundle offline and never calls Rekor per admission.

Verify at admission, not at build

Signing an image proves nothing on its own — the cluster has to check. That's Kyverno's job: an ImageValidatingPolicy intercepts every pod creation, resolves the image's signature, and verifies it against the known public key before the scheduler ever sees the pod.

apiVersion: policies.kyverno.io/v1alpha1
kind: ImageValidatingPolicy
metadata:
  name: verify-codeagency-images
spec:
  images:
    - glob: "zot.zot.svc.cluster.local:5000/codeagency/*"
  validationActions: [Audit]
  attestors:
    - name: cosign-key
      cosign:
        key:
          data: |
            -----BEGIN PUBLIC KEY-----
            ...

Two details matter more than the YAML shape. First, insecureIgnoreSCT: true — there's no SCT to check, because key-based signing never produced a Fulcio certificate in the first place; that flag isn't a weakening, it's just honest about the signing mode we chose. Second, the cache layer kaniko pushes alongside every build (web-cache) is explicitly excluded from the glob. It's never scheduled as a pod, so signing it would be ceremony with no security benefit.

Audit mode first, always

validationActions: [Audit] means the policy currently logs a violation, it doesn't block one. That's deliberate, not a placeholder we forgot to flip. A brand-new admission policy is a single YAML change with cluster-wide blast radius — the way you find out it has an unanticipated false positive should be a log line, not an outage. Audit mode runs for real, against real traffic, until the violation count sits at zero for long enough that flipping to Enforce is boring. We're not there yet, and we're not in a hurry to be: the failure mode of "the sign step hiccups and nothing deploys" is worse than "an unsigned image ran for an extra day," so deploy-gitops in the pipeline deliberately does not depend on sign — a signing failure logs, it never blocks the rollout.

The PR-preview exception

Every pull request on this repo gets its own live preview instance, built and deployed the same way production is — except the sign step only runs on: push, branch: main. PR images ship unsigned, on purpose.

That's not a hole in the policy, it's a scope decision: preview instances are ephemeral, torn down on merge or close, and never carry production traffic or secrets. Requiring a cosign key inside every PR build would mean either handing that key to untrusted branches or blocking previews on a step that has nothing to do with reviewing a feature. Narrowing the policy's glob to only match tags that reach production — plus running in Audit rather than Enforce — means an unsigned preview image never triggers anything a reviewer would notice, while main still gets signed on every single deploy.

What this actually buys us

A stolen CI token can still push an image. It can no longer make that image indistinguishable from one this pipeline actually built — the signature only exists if cosign sign ran with the real key, and the real key never leaves the signing step's environment. Combined with image freshness pinning and the same GitOps promotion path that already gates what reaches main, it's one more link in a chain where no single compromised credential is enough on its own. It's the same posture behind every cluster we run as part of managed cloud hosting — not because a compliance checklist demands it, but because the alternative is trusting a registry to be something it was never built to be.