---
title: Blue-green deploys with one-click rollback
slug: blue-green-deploys-with-rollback
summary: Deploy a candidate next to the live version, smoke-test it, promote on success, and keep the previous version one button away; a worked example on a container service and your CI provider.
order: 4
updated: 2026-11-11
status: ready
---

Use this page when designing or fixing a deployment pipeline, or when a deploy goes bad and you need to roll back. The pattern works on any platform: Kubernetes, a container service, VMs behind a load balancer, serverless aliases. The worked example uses a container service on AWS and a generic CI skeleton.

## The pattern in one picture

```mermaid
flowchart LR
  B[Build image once] --> D[Deploy candidate]
  D --> S{Smoke tests pass?}
  S -- no --> X[Delete candidate, live untouched]
  S -- yes --> R[Record current live version]
  R --> P[Promote candidate to live]
  P --> W{Health OK for N minutes?}
  W -- no --> RB[Rollback to recorded version]
  W -- yes --> C[Clean up candidate]
```

The rules that make it safe:

- [ ] **Build once, promote the same artifact.** Dev, staging and prod run the identical image digest. Never rebuild per environment.
- [ ] **The candidate gets real traffic only from tests** until it is promoted.
- [ ] **Record the previous version before promoting.** Rollback means "go back to the recorded version", not "guess".
- [ ] **Rollback is a single manual job** that needs no new build and no thinking.
- [ ] **Lower environments deploy automatically; production is a manual, protected step.**

## Environments and gates

| Environment | Trigger | Who can run it |
| --- | --- | --- |
| dev | automatic on merge to default branch | pipeline |
| staging or sandbox | manual job | developers |
| production | manual job, protected environment | release approvers only |
| production rollback | manual job, always available | on-call and release approvers |

## Worked example: ECS with a candidate service

Two ECS services per environment: `<service>` (live, behind the main target group) and `<service>-candidate` (behind a test listener or a separate target group that only the smoke tests hit).

1. Register a new task definition revision with the new image digest.
2. Update `<service>-candidate` to that revision and wait for it to be stable.
3. Run smoke tests against the candidate endpoint.
4. Store the live service's current task definition ARN, for example in a parameter `/deploy/<service>/<env>/previous`.
5. Update `<service>` to the new revision and wait for steady state.
6. Scale the candidate to zero.

```bash
# Read-only checks you can run any time
aws --profile <readonly-profile> --region <region> ecs describe-services \
  --cluster <cluster> --services <service> <service>-candidate \
  --query 'services[].{name:serviceName,td:taskDefinition,running:runningCount,desired:desiredCount}'

aws --profile <readonly-profile> --region <region> ssm get-parameter \
  --name /deploy/<service>/<env>/previous --query Parameter.Value --output text
```

Pipeline jobs use a deploy role assumed through [OIDC](./oidc-from-ci-to-cloud.md), never static keys.

## Worked example: a CI skeleton

```yaml
stages: [build, dev, sandbox, prod]

.deploy: &deploy
  image: <deploy-image>
  # Obtain a short-lived cloud role through OIDC here (see the OIDC page).
  script:
    - ./ci/deploy-candidate.sh "$ENV" "$IMAGE_DIGEST"
    - ./ci/smoke.sh "$ENV"
    - ./ci/promote.sh "$ENV"
    - ./ci/cleanup-candidate.sh "$ENV"

build:
  stage: build
  script:
    - ./ci/build-and-push.sh   # writes IMAGE_DIGEST to build.env
  artifacts:
    reports:
      dotenv: build.env

deploy_dev:
  <<: *deploy
  stage: dev
  variables: { ENV: dev }
  environment: { name: dev }
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy_sandbox:
  <<: *deploy
  stage: sandbox
  variables: { ENV: sandbox }
  environment: { name: sandbox }
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual

deploy_prod:
  <<: *deploy
  stage: prod
  variables: { ENV: prod }
  environment: { name: prod }
  needs: [build, deploy_sandbox]
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual

rollback_prod:
  stage: prod
  image: <deploy-image>
  variables: { ENV: prod }
  environment: { name: prod, action: access }
  script:
    - ./ci/rollback.sh "$ENV"   # reads /deploy/<service>/prod/previous and redeploys it
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual
  allow_failure: true
```

Mark `prod` as a protected environment in your CI so only approvers can press the button. Many platforms also offer a re-deploy of an earlier successful job, which is a second rollback path.

## Smoke tests that are worth running

- [ ] Health endpoint returns 200 and reports the expected version.
- [ ] One read path and one write path against test data succeed.
- [ ] Dependencies reachable: database, cache, queue, one downstream API.
- [ ] Fails fast: whole suite under 2 minutes, non-zero exit on any failure.

## Rollback runbook

- [ ] Open the pipeline for the last good production deploy, or the latest pipeline on the default branch.
- [ ] Run `rollback_prod`. It needs no inputs.
- [ ] Confirm the live service shows the previous task definition and is stable.
- [ ] Confirm error rate is back to baseline.
- [ ] Post in the incident channel and link the job.

## Alternatives

| Option | When it fits |
| --- | --- |
| ECS native blue/green or CodeDeploy | You want traffic shifting (canary, linear) and automatic rollback on alarms |
| Kubernetes Argo Rollouts or Flagger | Kubernetes, progressive delivery with metric analysis |
| Lambda aliases with weighted routing | Serverless functions |
| Feature flags | Decouple deploy from release for risky features |

## Pitfalls

- Database migrations that are not backward compatible. Expand first, deploy, contract later, or rollback breaks.
- Rebuilding the image in the prod job. Promote the digest you tested.
- Storing "previous" only in the pipeline log. Store it somewhere the rollback job reads.
- A rollback job that itself needs the build stage. It must run on its own.