Intermediateobservability
The Log Tsunami
Your centralized logging pipeline (Fluentd → Elasticsearch) is backed up. A microservice started debug logging in a hot loop, producing 50,000 logs/second. Elasticsearch is rejecting bulk inserts, Fluentd buffers are full, and now ALL services are losing logs — including the ones you need for an active P1 investigation.
INCOMING INCIDENT
$ incident --describe the-log-tsunami
Your centralized logging pipeline (Fluentd → Elasticsearch) is backed up. A microservice started debug logging in a hot loop, producing 50,000 logs/second. Elasticsearch is rejecting bulk inserts, Fluentd buffers are full, and now ALL services are losing logs — including the ones you need for an active P1 investigation.
4
Decision Points
60s
Per Step
400
Max Score
200
Passing Score