# Alerts That Earn the Page

> If nothing can be done at 3am, it is not a page. How to build a pager people trust.

Source: https://christwentyfour.com/blog/alerts-that-earn-the-page

If an alert fires at 3am and no human can act on it, it is not a page. It can be a ticket, a dashboard panel, or a weekly report. It should not wake anyone.

Most bad pages alert on causes. CPU is high. Disk is at 80 percent. A pod restarted. Sometimes those are fine. A service can run at 90 percent CPU all day.

## Page on what users feel

Page on failing requests, latency past the promise you made, or a queue that is growing. That list is short and stable. Causes belong on the dashboard you open after the page.

Every page needs three things: what broke in user terms, where to look, and what to do first. If you cannot write the first step, you do not understand the failure well enough to page on it.

## Measure how often a page was worth it

$$\text{alert precision} = \frac{\text{pages that needed action}}{\text{total pages}}$$

**Worked example:** **30%** alert precision

If 9 of last month's 30 pages needed a human, precision is 30%. The other 21 woke someone for nothing.

## Delete alerts on purpose

Once a quarter, sort the pages by frequency and read the top ten. If the honest answer is "acknowledged it and went back to sleep," delete or downgrade it. Alert fatigue is how real pages get missed.
The pager should be quiet most weeks.