Change failure rate is one of the four delivery metrics worth tracking, and it is the one most easily gamed without anybody intending to game it. The definition sounds unambiguous: the proportion of deployments that cause a degradation requiring remediation. In practice, every clause in that sentence is doing negotiable work.
What counts as a failure
If a bad release is caught by a canary and rolled back automatically in ninety seconds, was that a failure? By a strict reading, yes: the change degraded the service and required remediation. By the reading most teams drift toward, no, because the system handled it and nobody was paged.
Both answers are defensible, but they measure very different things, and a team that quietly moves from the first definition to the second will show a falling failure rate while nothing has actually improved. Worse, the definition tends to loosen precisely when the number is under scrutiny.
My preference is the strict reading, because it keeps the number honest about how often changes are wrong, which is the thing you want to know. The fast automated rollback then shows up where it belongs, in the time-to-restore figure.
A low rate can mean you have stopped shipping
The metric is a ratio, and the denominator matters. A team deploying once a month with a five percent failure rate is not outperforming a team deploying twenty times a week at eight percent. The first team has fewer failures because it has fewer changes, and each of those changes is enormous, which is its own risk that the metric does not capture.
This is why the four metrics have to be read together. Any one of them can be improved by damaging another, and the only meaningful claim is about all of them at once. The research finding worth holding onto is that the best teams do well on all four simultaneously, which means a trade-off between them is usually a sign of a problem rather than a strategy.
It says nothing about severity
Ten deployments that each caused a two-minute blip look identical to ten that each took down checkout for an hour. If you are using the number to decide where to invest, that is a significant blind spot.
Pairing it with something severity-weighted, even crudely, tends to be more actionable. Total minutes of degradation attributable to changes is not a standard metric but it is often the one that tells you whether the last quarter went well.
Use it as a trend, not a target
The moment a delivery metric becomes a target with consequences attached, it stops measuring anything. People will classify differently, batch differently, and describe incidents differently, all without deciding to be dishonest.
Kept as an internal trend, looked at quarterly, and interpreted alongside the other three, it is genuinely useful: a rising rate is an early sign that something in the pipeline, the test suite, or the review process has degraded, and that is worth knowing before it becomes obvious some other way.