Skip to content

Deployment Strategies

What You'll Learn

  • How RollingUpdate and Recreate actually behave, pod by pod
  • How to tune maxSurge and maxUnavailable for your availability and cost trade-off
  • How kubectl rollout status/history/undo fit into a real release, and where blue-green and canary sit relative to a plain Deployment
  • How PodDisruptionBudgets keep a workload available during node drains and cluster upgrades

Why This Matters

A Deployment's strategy field is the difference between "users never notice a release happened" and "half your requests 503 for ninety seconds." Most production incidents from deployments aren't caused by bad code — they're caused by a rollout strategy nobody tuned past the default.

Mental Model

A Deployment doesn't update pods directly. It manages ReplicaSets, and a rollout is really: create a new ReplicaSet, scale it up, scale the old one down, according to the strategy. RollingUpdate (the default) does this gradually and keeps the app available throughout; Recreate kills every old pod before starting any new one.

Strategy Old pods removed New pods created Downtime Use when
RollingUpdate Gradually, as new pods pass readiness Gradually, up to maxSurge over desired count None (if probes are correct) Default — almost everything stateless
Recreate All at once, before any new pod starts Only after all old pods are gone Yes, by design The app can't run two versions at once (schema-incompatible singleton, exclusive port/lock)

RollingUpdate, tuned

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout-api
spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2          # up to 8 pods running briefly
      maxUnavailable: 0    # never drop below 6 ready pods
  minReadySeconds: 10       # pod must stay Ready this long before counting as "up"
  selector:
    matchLabels:
      app: checkout-api
  template:
    metadata:
      labels:
        app: checkout-api
    spec:
      containers:
        - name: checkout-api
          image: registry.example.com/checkout-api:1.14.2
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /healthz/ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 5
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 500m
              memory: 512Mi
  • maxSurge — how many pods above replicas can exist during the rollout. Higher = faster rollout, more capacity needed.
  • maxUnavailable — how many pods below replicas are tolerated during the rollout. 0 guarantees full capacity throughout, at the cost of needing headroom for maxSurge extra pods.
  • Setting both to sensible non-zero values (e.g. maxSurge: 25%, maxUnavailable: 25%, the actual defaults) balances rollout speed against cluster headroom.
  • A rollout only proceeds pod-by-pod as fast as the readiness probe says the new pods are healthy — a broken readiness probe silently stalls or fast-tracks a bad rollout.

Percentages are resolved against replicas with the same rules the Deployment controller uses: maxSurge rounds up, maxUnavailable rounds down. Try your own values:

Rolling update calculator (needs JavaScript). With 10 replicas and the default 25%/25%, maxSurge resolves to 3 and maxUnavailable to 2: at most 13 Pods exist at once, and at least 8 stay available.

Managing a rollout

kubectl apply -f checkout-api-deployment.yaml
kubectl rollout status deployment/checkout-api
kubectl rollout history deployment/checkout-api
kubectl rollout history deployment/checkout-api --revision=3
kubectl rollout undo deployment/checkout-api
kubectl rollout undo deployment/checkout-api --to-revision=2
kubectl rollout restart deployment/checkout-api

rollout status blocks until the rollout finishes or fails — use it in CI to gate the next pipeline step. rollout undo works because Kubernetes keeps old ReplicaSets around (revisionHistoryLimit, default 10) — undo just scales the previous ReplicaSet back up and the current one down, which is itself a RollingUpdate.

flowchart LR
    A[Old ReplicaSet: 6/6 ready] -->|apply new image| B[New ReplicaSet scales up]
    B -->|maxSurge allows +2| C[8 pods total, new ones become Ready]
    C -->|old pods scaled down| D[Old ReplicaSet: 0/0]
    D --> E[New ReplicaSet: 6/6 ready]
    E -->|rollout undo| A

PodDisruptionBudgets: Availability During Maintenance

A rollout is a disruption you start. Node drains, cluster upgrades, Cluster Autoscaler scale-downs, and spot reclaims are disruptions the platform starts, and a Deployment's maxUnavailable doesn't apply to them. A PodDisruptionBudget (PDB) is how a workload tells the cluster how many of its Pods may be taken down voluntarily at once:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-api
spec:
  minAvailable: 5            # or: maxUnavailable: 1
  selector:
    matchLabels:
      app: checkout-api
  unhealthyPodEvictionPolicy: AlwaysAllow   # let drains evict Pods that are already broken

kubectl drain, managed node upgrades, and the autoscalers all use the Eviction API, which checks every matching PDB. If evicting a Pod would drop the workload below minAvailable, the eviction is refused and retried until another replica is ready elsewhere. With 6 replicas and minAvailable: 5, nodes drain one checkout Pod at a time.

kubectl get pdb -n production
# NAME           MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
# checkout-api   5               N/A               1                     12d

ALLOWED DISRUPTIONS is the number to watch. If it's 0 for long, drains and upgrades stall on this workload.

Guideline Why
Prefer maxUnavailable: 1 for most Deployments Keeps working as replica counts change; minAvailable pinned to a number can block all drains after a scale-down
Never set a PDB that allows zero disruptions (minAvailable equal to replicas, or maxUnavailable: 0) Every node drain and cluster upgrade hangs on it forever
Single-replica workloads can't be protected by a PDB The only way to keep them available during maintenance is to run a second replica
PDBs don't cover involuntary disruptions A node crash or kernel panic ignores PDBs; replicas spread across nodes and zones are what protect against those
Set unhealthyPodEvictionPolicy: AlwaysAllow Otherwise a crash-looping Pod counts against the budget and blocks the drain that might fix it

Blue-Green and Canary, Conceptually

RollingUpdate mixes old and new pods behind the same Service the entire time — fine for most apps, but it means both versions serve traffic simultaneously and you can't instantly cut back to 100% old version if something's wrong.

  • Blue-green runs two full environments (blue = current, green = new) and switches traffic all at once, usually by repointing a Service selector or load balancer. Rollback is instant — flip back to blue — but you pay for two full environments during the switch.
  • Canary sends a small percentage of traffic to the new version, watches error rates/latency, then ramps up gradually. It catches bad releases with minimal blast radius, but needs traffic-splitting infrastructure a plain Deployment doesn't have.

A Deployment alone can approximate both crudely (two Deployments + a Service selector swap for blue-green; two Deployments with proportional replica counts for a rough canary), but real weighted traffic splitting, automated analysis, and automated rollback need a progressive-delivery tool.

Where the full walkthroughs live

Operational, tool-based walkthroughs with Argo Rollouts and Flagger are in Progressive Delivery: Canary and Blue-Green. A real-world postmortem of both patterns is in Blue-Green and Canary Releases.

What Most Engineers Miss

  • The two percentages round in opposite directions. maxSurge rounds up and maxUnavailable rounds down, so with 3 replicas the default 25%/25% means up to 1 extra Pod and 0 unavailable. Small Deployments roll one surge Pod at a time.
  • If both round to zero, the controller quietly allows one unavailable Pod. The API rejects maxSurge: 0 with maxUnavailable: 0, but maxSurge: 0% with maxUnavailable: 25% on 3 replicas rounds both to 0, and the controller then sets maxUnavailable to 1. A config meant to never drop capacity takes a Pod out.
  • A stuck rollout is reported, not rolled back. After progressDeadlineSeconds (default 600), the Deployment's Progressing condition turns False with reason ProgressDeadlineExceeded, and that's all Kubernetes does: the new Pods stay. Your pipeline has to watch for it and run kubectl rollout undo, or use Argo Rollouts or Flagger.
  • Terminating Pods can still receive traffic. Removing a Pod from Service endpoints and sending it SIGTERM happen at the same time, so load balancers and kube-proxy can route a few more requests to a Pod that's shutting down. A short preStop delay closes the gap, and since v1.34 the built-in sleep action does it without a sleep binary in the image:

    lifecycle:
      preStop:
        sleep:
          seconds: 5
    
  • minReadySeconds defaults to 0. A Pod that turns Ready and crashes two seconds later has already been counted as available, and the rollout has moved on. Set it to a few seconds longer than your app's typical crash-on-start time.

  • Undo only reaches back revisionHistoryLimit revisions (default 10). Set it to 0 to save a few objects, and kubectl rollout undo has nothing to go back to.

Common Mistakes

  • Setting maxUnavailable: 0 and maxSurge: 0 at the same time — the API rejects the Deployment (may not be 0 when maxSurge is 0), because a rollout with no room to add or remove Pods could never progress.
  • No readiness probe (or a probe that always passes) — the rollout has no real signal, so a broken new pod gets marked ready and takes production traffic immediately.
  • Using Recreate out of habit "to be safe" — it introduces downtime a RollingUpdate with maxUnavailable: 0 doesn't need.
  • Confusing kubectl rollout restart (re-rolls current pods with the same spec, e.g. to pick up a changed Secret) with kubectl rollout undo (reverts to a previous revision).
  • Assuming a successful rollout status means the release is actually healthy — it only means pods are Ready per the probe; it says nothing about real user-facing error rates.

Check Yourself

Defaults on a small Deployment

A Deployment has replicas: 3 and the default strategy (maxSurge: 25%, maxUnavailable: 25%). What do the two limits resolve to?

  • Surge 0, unavailable 0
  • Surge 1, unavailable 0
  • Surge 1, unavailable 1

25% of 3 is 0.75. maxSurge rounds up to 1, maxUnavailable rounds down to 0, so the rollout adds one new Pod, waits for it to be Ready, then removes an old one.

Past the deadline

A new image crashes on start, and the rollout exceeds progressDeadlineSeconds. What does Kubernetes do?

  • Rolls back to the previous ReplicaSet automatically
  • Marks the Deployment Progressing=False (ProgressDeadlineExceeded) and leaves everything as it is
  • Deletes the new ReplicaSet

Kubernetes only reports a stalled rollout. Rolling back is up to you, your pipeline, or a progressive delivery controller.

Rounded to zero

replicas: 4, maxSurge: 0, maxUnavailable: 10%. How many Pods can be unavailable during the rollout?

  • 0, so the rollout hangs
  • 1: 10% of 4 rounds down to 0, and with both limits at 0 the controller uses 1
  • The API rejects the Deployment

The API only rejects literal zeros for both. When rounding produces zero for both, the controller allows one unavailable Pod so the rollout can move.

Interview Questions

  • Walk through exactly what happens, pod by pod, during a RollingUpdate with maxSurge: 1, maxUnavailable: 0.
  • A cluster upgrade is stuck draining one node. How would a PodDisruptionBudget cause that, and how do you find which one?
  • Why would you choose Recreate over RollingUpdate, and what's the cost?
  • How does kubectl rollout undo work under the hood, given Deployments don't store "old pods" directly?
  • How does a canary release differ from what a Deployment's RollingUpdate already gives you?

See Interview Prep for full answers.

Next

Continue to StatefulSets for workloads that need stable identity instead of interchangeable pods.