Intermediateobservability
The Alert Storm
Your on-call phone won't stop buzzing. Prometheus is firing 200+ alerts per minute. PagerDuty is auto-escalating to management. Half the alerts are false positives from a network blip that resolved itself 10 minutes ago. But buried in the noise, there's a real production issue. Find it.
INCOMING INCIDENT
$ incident --describe the-alert-storm
Your on-call phone won't stop buzzing. Prometheus is firing 200+ alerts per minute. PagerDuty is auto-escalating to management. Half the alerts are false positives from a network blip that resolved itself 10 minutes ago. But buried in the noise, there's a real production issue. Find it.
5
Decision Points
60s
Per Step
500
Max Score
300
Passing Score