Deployment Strategies¶
What You'll Learn¶
- How
RollingUpdateandRecreateactually behave, pod by pod - How to tune
maxSurgeandmaxUnavailablefor your availability and cost trade-off - How
kubectl rollout status/history/undofit into a real release, and where blue-green and canary sit relative to a plain Deployment - How PodDisruptionBudgets keep a workload available during node drains and cluster upgrades
Why This Matters¶
A Deployment's strategy field is the difference between "users never notice a release happened" and "half your requests 503 for ninety seconds." Most production incidents from deployments aren't caused by bad code — they're caused by a rollout strategy nobody tuned past the default.
Mental Model¶
A Deployment doesn't update pods directly. It manages ReplicaSets, and a rollout is really: create a new ReplicaSet, scale it up, scale the old one down, according to the strategy. RollingUpdate (the default) does this gradually and keeps the app available throughout; Recreate kills every old pod before starting any new one.
| Strategy | Old pods removed | New pods created | Downtime | Use when |
|---|---|---|---|---|
RollingUpdate |
Gradually, as new pods pass readiness | Gradually, up to maxSurge over desired count |
None (if probes are correct) | Default — almost everything stateless |
Recreate |
All at once, before any new pod starts | Only after all old pods are gone | Yes, by design | The app can't run two versions at once (schema-incompatible singleton, exclusive port/lock) |
RollingUpdate, tuned¶
apiVersion: apps/v1
kind: Deployment
metadata:
name: checkout-api
spec:
replicas: 6
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 2 # up to 8 pods running briefly
maxUnavailable: 0 # never drop below 6 ready pods
minReadySeconds: 10 # pod must stay Ready this long before counting as "up"
selector:
matchLabels:
app: checkout-api
template:
metadata:
labels:
app: checkout-api
spec:
containers:
- name: checkout-api
image: registry.example.com/checkout-api:1.14.2
ports:
- containerPort: 8080
readinessProbe:
httpGet:
path: /healthz/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
maxSurge— how many pods abovereplicascan exist during the rollout. Higher = faster rollout, more capacity needed.maxUnavailable— how many pods belowreplicasare tolerated during the rollout.0guarantees full capacity throughout, at the cost of needing headroom formaxSurgeextra pods.- Setting both to sensible non-zero values (e.g.
maxSurge: 25%,maxUnavailable: 25%, the actual defaults) balances rollout speed against cluster headroom. - A rollout only proceeds pod-by-pod as fast as the readiness probe says the new pods are healthy — a broken readiness probe silently stalls or fast-tracks a bad rollout.
Managing a rollout¶
kubectl apply -f checkout-api-deployment.yaml
kubectl rollout status deployment/checkout-api
kubectl rollout history deployment/checkout-api
kubectl rollout history deployment/checkout-api --revision=3
kubectl rollout undo deployment/checkout-api
kubectl rollout undo deployment/checkout-api --to-revision=2
kubectl rollout restart deployment/checkout-api
rollout status blocks until the rollout finishes or fails — use it in CI to gate the next pipeline step. rollout undo works because Kubernetes keeps old ReplicaSets around (revisionHistoryLimit, default 10) — undo just scales the previous ReplicaSet back up and the current one down, which is itself a RollingUpdate.
flowchart LR
A[Old ReplicaSet: 6/6 ready] -->|apply new image| B[New ReplicaSet scales up]
B -->|maxSurge allows +2| C[8 pods total, new ones become Ready]
C -->|old pods scaled down| D[Old ReplicaSet: 0/0]
D --> E[New ReplicaSet: 6/6 ready]
E -->|rollout undo| A
PodDisruptionBudgets: Availability During Maintenance¶
A rollout is a disruption you start. Node drains, cluster upgrades, Cluster Autoscaler scale-downs, and spot reclaims are disruptions the platform starts, and a Deployment's maxUnavailable doesn't apply to them. A PodDisruptionBudget (PDB) is how a workload tells the cluster how many of its Pods may be taken down voluntarily at once:
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: checkout-api
spec:
minAvailable: 5 # or: maxUnavailable: 1
selector:
matchLabels:
app: checkout-api
unhealthyPodEvictionPolicy: AlwaysAllow # let drains evict Pods that are already broken
kubectl drain, managed node upgrades, and the autoscalers all use the Eviction API, which checks every matching PDB. If evicting a Pod would drop the workload below minAvailable, the eviction is refused and retried until another replica is ready elsewhere. With 6 replicas and minAvailable: 5, nodes drain one checkout Pod at a time.
kubectl get pdb -n production
# NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
# checkout-api 5 N/A 1 12d
ALLOWED DISRUPTIONS is the number to watch. If it's 0 for long, drains and upgrades stall on this workload.
| Guideline | Why |
|---|---|
Prefer maxUnavailable: 1 for most Deployments |
Keeps working as replica counts change; minAvailable pinned to a number can block all drains after a scale-down |
Never set a PDB that allows zero disruptions (minAvailable equal to replicas, or maxUnavailable: 0) |
Every node drain and cluster upgrade hangs on it forever |
| Single-replica workloads can't be protected by a PDB | The only way to keep them available during maintenance is to run a second replica |
| PDBs don't cover involuntary disruptions | A node crash or kernel panic ignores PDBs; replicas spread across nodes and zones are what protect against those |
Set unhealthyPodEvictionPolicy: AlwaysAllow |
Otherwise a crash-looping Pod counts against the budget and blocks the drain that might fix it |
Blue-Green and Canary, Conceptually¶
RollingUpdate mixes old and new pods behind the same Service the entire time — fine for most apps, but it means both versions serve traffic simultaneously and you can't instantly cut back to 100% old version if something's wrong.
- Blue-green runs two full environments (blue = current, green = new) and switches traffic all at once, usually by repointing a Service selector or load balancer. Rollback is instant — flip back to blue — but you pay for two full environments during the switch.
- Canary sends a small percentage of traffic to the new version, watches error rates/latency, then ramps up gradually. It catches bad releases with minimal blast radius, but needs traffic-splitting infrastructure a plain Deployment doesn't have.
A Deployment alone can approximate both crudely (two Deployments + a Service selector swap for blue-green; two Deployments with proportional replica counts for a rough canary), but real weighted traffic splitting, automated analysis, and automated rollback need a progressive-delivery tool.
Where the full walkthroughs live
Operational, tool-based walkthroughs with Argo Rollouts and Flagger are in Progressive Delivery: Canary and Blue-Green. A real-world postmortem of both patterns is in Blue-Green and Canary Releases.
Common Mistakes¶
- Setting
maxUnavailable: 0andmaxSurge: 0at the same time — the rollout can never make progress and hangs. - No readiness probe (or a probe that always passes) — the rollout has no real signal, so a broken new pod gets marked ready and takes production traffic immediately.
- Using
Recreateout of habit "to be safe" — it introduces downtime aRollingUpdatewithmaxUnavailable: 0doesn't need. - Confusing
kubectl rollout restart(re-rolls current pods with the same spec, e.g. to pick up a changed Secret) withkubectl rollout undo(reverts to a previous revision). - Assuming a successful
rollout statusmeans the release is actually healthy — it only means pods are Ready per the probe; it says nothing about real user-facing error rates.
Interview Questions¶
- Walk through exactly what happens, pod by pod, during a
RollingUpdatewithmaxSurge: 1, maxUnavailable: 0. - A cluster upgrade is stuck draining one node. How would a PodDisruptionBudget cause that, and how do you find which one?
- Why would you choose
RecreateoverRollingUpdate, and what's the cost? - How does
kubectl rollout undowork under the hood, given Deployments don't store "old pods" directly? - How does a canary release differ from what a Deployment's
RollingUpdatealready gives you?
See Interview Prep for full answers.
Related¶
Next¶
Continue to StatefulSets for workloads that need stable identity instead of interchangeable pods.