Cloud spend usually lives with finance, or with whoever drew the short straw, and it is treated as an efficiency problem. That framing misses something useful: a sudden change in cost is often the earliest available signal that something is wrong technically, and it frequently arrives before any of your monitoring notices.
What a cost spike is telling you
Bills go up for boring reasons, like a deliberate scale-up. They also go up for reasons worth investigating immediately:
- A retry loop that has started firing constantly, so request volume is real but pointless.
- A log line added at debug level in a hot path, now generating far more ingest than the service generates value.
- An autoscaler compensating for a performance regression, quietly buying its way around a bug.
- Data transfer between zones that used to be within one, because something moved.
- Storage growing without bound because a lifecycle rule was never set, or was set and then broken.
Every one of those is a technical defect that presents first as money. The autoscaler case is the most insidious, because the system stays up and the graphs look fine. You are paying to hide a regression, and you will keep paying until someone looks at the bill.
Alert on the derivative
Monthly review is too slow. By the time the invoice arrives, you have paid for a month of whatever it is. What works is an alert on daily spend deviating from the trailing average by some margin, per service if your tagging supports it.
This does not need to be sophisticated. A simple threshold catches most of the interesting cases, and the false positives are cheap: someone looks, says "yes, we launched that," and moves on.
Tagging is the prerequisite
None of this works if you cannot attribute spend. Untagged resources become an ever-growing bucket labelled "other," and once it is large enough nobody can tell whether a change inside it matters.
Enforcing tags at creation time, in the infrastructure code rather than by policy reminder, is the only approach I have seen hold. Retroactive tagging projects are miserable and do not stay done.
Right-sizing is a reliability activity too
The usual framing of right-sizing is savings. But an instance that is dramatically oversized is also hiding information: you do not know how the service behaves under pressure because it never experiences any. When you eventually do need to scale, you will be doing it for the first time, during an event.
Conversely, a service running close to its limits to save money has no headroom for the surprise. The right size is the one where you understand the behaviour on both sides, and cost is one of the inputs to that rather than the only one.
The organizational version
Making spend visible per team changes behavior more than any policy does. Not as a target to punish, but as information people did not previously have. Most engineers have no idea what their service costs, and most of them care once they find out.