# What Change Failure Rate Hides

> The delivery metric that is easiest to game without meaning to.

Source: https://christwentyfour.com/blog/what-change-failure-rate-hides

Change failure rate is one of four delivery metrics. It is also the easiest to game without meaning to. Each part of its definition leaves room to interpret.

## What counts as a failure

Say a canary catches a bad release and rolls it back in ninety seconds. Was that a failure? Strictly, yes. Most teams drift toward no, because the system handled it. A team that drifts from strict to lenient will see the rate fall while nothing improves. I use the strict reading. Fast rollback then shows up in time to restore.

$$\text{change failure rate} = \frac{\text{changes that caused a failure}}{\text{total changes}}$$

**Worked example:** **8%** change failure rate

Four failed changes out of 50 is 8%.

## The denominator matters

A team that deploys monthly at 5 percent failure is not beating a team that deploys weekly at 8 percent. The first makes fewer, larger changes, which is its own risk.

## Read the four together

Any one metric can be improved by hurting another. The best teams do well on all four at once.

## It ignores severity

Ten two-minute blips look the same as ten hour-long outages. Pair the rate with total minutes of change-caused degradation.

**Worked example:** **120** minutes of degradation

Fifty changes at 8%, with 30 minutes to restore each, is \(50 \times 0.08 \times 30 = 120\) minutes.

## Use it as a trend, not a target

A target invites people to classify differently without deciding to be dishonest. Reviewed quarterly, a rising rate is an early warning.