# Containers & ECS

> A container gets its config at startup and reaches out for nothing else.

Source: https://christwentyfour.com/runbooks/containers

## The sealed package

Most container problems I have debugged in production were not container problems. The app reached out at startup and got a different answer than it did in test.

Containers should receive their configuration like a sealed package.

Everything arrives as environment variables or injected secrets. The app reads them. It does not fetch them.

A container that never calls the cloud API at startup cannot fail on a throttled API, a missing boot-time permission, or a slow regional endpoint.

## Configuration comes in

The task definition resolves secrets, not the app's startup code.

**task definition: secrets injected by the platform**

```text
{
  "name": "api",
  "image": "…/api@sha256:9f2c…",
  "environment": [
    { "name": "LOG_LEVEL", "value": "info" }
  ],
  "secrets": [
    { "name": "DB_PASSWORD",
      "valueFrom": "arn:aws:secretsmanager:…:secret:prod/api/db" }
  ]
}
```

The app sees `DB_PASSWORD` as a plain environment variable. It does not know Secrets Manager exists.

**Do:**

- Inject secrets through the task definition's `secrets` block.
- Keep KMS out of application code.
- Scope the task role to what the app needs. Usually very little.
- Pin images by digest, not `latest`.

**Don't:**

- No SDK calls for config at startup: no SSM, no KMS, no STS.
- No per-environment config baked into the image.

## If it needs cloud credentials to boot, it is not portable

Ask if a developer can run the service on a laptop with no AWS credentials, using a `.env` file. If not, the app has a hidden dependency on its runtime. That causes bugs that only reproduce in one environment, the most expensive kind to chase.

## Image size is a delivery metric

Multi-gigabyte images are usually accidental. Every pipeline run pushes them, and every scale event pulls them.

**Dockerfile: build stage discarded**

```text
FROM node:22-slim AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci                      # dev deps live and die here
COPY . .
RUN npm run build

FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev && npm cache clean --force
COPY --from=build /app/dist ./dist
USER node                      # never run as root
CMD ["node", "dist/server.js"]
```

Put dependency layers before source. A code change should not bust the dependency cache. That is often worth more than the size cut.

## Blue/green, and rollback you have tested

Blue/green keeps the previous version running, so rollback is a traffic shift, not a pipeline run. The part teams skip is the health check. `/health` returning 200 because the web server started tells you almost nothing.

**Do:**

- Make the health check verify the dependencies the service needs.
- Set a deregistration delay for in-flight requests.
- Test the rollback on purpose. An untested rollback is a hypothesis.

Blue/green chooses which *version* serves. Feature flags choose which *users* see new behavior.

## Anti-patterns

**Don't:**

- **Running as root.** One line to fix. It is the first thing a scanner flags.
- **`latest` in a task definition.** You cannot say what is running.
- **Secrets as plain `environment` values.** Use `secrets`. Plaintext ends up in revision history.
- **One image per environment.** You tested a different artifact than you shipped.
- **Vertical scaling as an architecture fix.** Cap the instance size. That exposes the real problem.

## With Claude

### Auditing a Dockerfile

Name the standard. "Is this good?" gets generic advice.

**prompt**

```text
Review this Dockerfile as a container security and build-performance
audit. Structure the output as a table: finding, severity, fix.

Check specifically:
  - runs as non-root
  - multi-stage, with no build toolchain in the final image
  - layer order maximizes cache reuse across code-only changes
  - no secrets or credentials in any layer (including intermediates)
  - base image is pinned and currently supported

Ignore style preferences. Only report things with a concrete impact.
```

### Finding startup-time cloud calls

**prompt**

```text
Search this service for configuration being fetched at startup rather
than injected: SSM parameter reads, Secrets Manager calls, KMS
decrypts, STS assume-role, any SDK client constructed at import time.

For each: file and line, what it fetches, and what it would take to
move it into the task definition instead.

I want to know whether this service can boot with no AWS credentials.
Answer that question explicitly at the end.
```

### Why a task won't start

Give the model the stopped-task reason, the task definition, and the log tail together. The answer is usually in how they interact.

Let it read. Be careful what it can change.

Registering a task definition against production cannot be undone in five minutes. A person does that.