CloudNativePG in production: Postgres on Kubernetes without fear

"Don't run your database on Kubernetes" was good advice for a long time — StatefulSets give you stable identity and stable storage, but nothing about failover, backups or version upgrades. Wire that up yourself with a sidecar and a prayer and you'll eventually pay for the shortcut. CloudNativePG (CNPG) is the reason we stopped agreeing with that advice: an operator that turns "primary died, promote a replica" and "restore from last night" into declarative Kubernetes resources instead of runbooks. Four years and counting under Odoo instances and plain custom web applications later, here's what that actually bought us.

A cluster is a YAML file, not a checklist

The whole pitch fits in one CRD. You declare the shape you want; the operator reconciles reality toward it, the same way a Deployment reconciles pods:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: erp-postgres
spec:
  instances: 3
  postgresql:
    parameters:
      max_connections: "200"
      shared_buffers: "2GB"
  storage:
    storageClass: longhorn
    size: 100Gi
  backup:
    barmanObjectStore:
      destinationPath: "s3://backups/erp-postgres"
      wal:
        compression: gzip
    retentionPolicy: "30d"

instances: 3 is one primary and two streaming replicas, synced via physical replication the operator wires up itself — no manual pg_basebackup, no hand-edited recovery.conf. Bump that number and CNPG provisions a new replica, streams it up to date, and adds it to the pool. Nothing here is CNPG-specific SQL or a proprietary dialect — it's the Postgres you already know, described declaratively.

Failover that doesn't wake anyone up

This is the feature that justifies the whole exercise. The operator runs its own consensus layer to decide which pod is the true primary — it doesn't trust pg_isready on one node in isolation, it needs agreement across the cluster before it promotes anything. When the primary pod dies — node drain, OOM, a bad kubectl delete — CNPG:

1. Detects the primary is gone via its status watchdog, not a naive liveness probe.
2. Picks the replica with the most caught-up WAL position.
3. Promotes it and repoints the read-write service — erp-postgres-rw — at the new primary.
4. Rebuilds the old primary as a new replica once it comes back, instead of leaving it as an orphaned split-brain risk.

Applications never talk to a pod name or an IP — they talk to the -rw and -ro services CNPG manages, so a promotion is invisible below the connection-string layer. In four years of client incidents, actual data-loss failovers: zero. Failovers where nobody manually intervened: all of them. That's the bar "production-ready" needs to clear, and it's also exactly the property our Odoo-on-Kubernetes stack leans on — the ERP doesn't know or care that its database pod restarted somewhere else.

Backups you don't have to remember to run

We've written before about treating a backup you've never restored as wishful thinking. CNPG's barmanObjectStore config turns that discipline into a spec instead of a cron job someone has to maintain:

backup:
  barmanObjectStore:
    destinationPath: "s3://backups/erp-postgres"
    wal:
      compression: gzip
      maxParallel: 4
  retentionPolicy: "30d"

Every WAL segment ships to object storage as it's generated — that's continuous archiving, not a nightly pg_dump. Point-in-time recovery follows from that for free:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: erp-postgres-restored
spec:
  instances: 1
  bootstrap:
    recovery:
      source: erp-postgres
      recoveryTarget:
        targetTime: "2026-07-30 09:14:00+00"
  externalClusters:
    - name: erp-postgres
      barmanObjectStore:
        destinationPath: "s3://backups/erp-postgres"

We still run the monthly restore drill — a config that's never been exercised is still a guess — but the drill now proves the pipeline works rather than a hand-rolled dump script. That distinction mattered exactly once, when a client's storage-layer snapshot silently stopped completing for a week. WAL archiving to S3 never depended on that snapshot job, so recovery was a non-event.

Major upgrades without a maintenance window nobody wants

Postgres major-version upgrades are traditionally the outage everyone schedules for a Sunday night. CNPG's answer is logical replication into a new cluster running the target version, cut over when it's caught up:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: erp-postgres-17
spec:
  instances: 3
  imageName: ghcr.io/cloudnative-pg/postgresql:17.4
  bootstrap:
    initdb:
      import:
        type: monolith
        databases: ["erp"]
        source:
          externalCluster: erp-postgres-16

The old cluster keeps serving traffic while the new one imports and catches up; you flip the application's connection string once the two are in sync, and roll back by flipping it again if something's wrong. We've done three major-version jumps this way. All three were scheduled during business hours, because there was no reason not to.

Gotchas to plan for, not bugs to fear

CNPG doesn't hide the fact that Postgres is a stateful workload — it surfaces the sharp edges immediately instead of papering over them. None of the following are operator bugs; they're the operational discipline any production Postgres deserves on Kubernetes, and worth building in from day one rather than discovering under load:

• Connection storms after a promotion. A failover briefly closes every connection at once, and an application with thin default connection pooling (Odoo included) can turn that into a max_connections spike on the new primary. Put PgBouncer in front via CNPG's own Pooler resource from the start, rather than treating it as optional.
• Storage class matters more than the operator. A CSI driver with slow volume attach/detach can leave a promoted replica waiting a minute or more just to get its disk reattached before Postgres can even begin recovery. We standardize on Longhorn for exactly this reason — attach times land in single-digit seconds. CNPG can only fail over as fast as the disk underneath it lets it.
• shared_buffers on a shared node pool. Left at the operator's conservative default, a co-scheduled noisy neighbor can starve a database pod during a load spike. Size shared_buffers explicitly and set CPU/memory requests equal to limits on database pods — guaranteed QoS, not best-effort.

What we'd still reach for something else

Postgres-as-everything is a stance we hold generally, and CNPG is what makes it hold up under Kubernetes specifically — but not universally. A single-node analytics warehouse doing multi-terabyte scans is a different physics problem; a team with zero Kubernetes operational experience and one production database is often better served by a managed offering where someone else owns the storage layer. CNPG earns its keep once you're already running Kubernetes for the application tier and want the database to be a first-class citizen of the same platform, on the same cluster — not a special pet living outside it.

Four years, three major-version upgrades, and a handful of failovers nobody had to page for later: the fear the old advice was protecting you from is largely gone. What's left is ordinary operational care — the same care any stateful system deserves, whether it announces itself with a YAML manifest or not.