Whenever I see a product this large having an outage (a lot recently, especially with Facebook, GCP, and AWS), I can only think of how stressful it must be for whoever needs to fix it. I've been on the other side of the outage before, albeit with a much smaller product, but lord was that stressful. Thinking about the random engineers that are stressed out and thinking that they could be fired for this, even if the ca…
Response for this incident went by the book, as described in Brent's talk above. Incident Management programs like these ensure that incidents can be resolved while also minimizing stress and chaos for engineers and other responders.
PagerDuty has a good Incident Responder and Incident Commander training courses, if you are interested in setting up a program similar to Slack's:
- https://response.pagerduty.com/training/courses/incident_res...
- https://response.pagerduty.com/training/incident_commander/
Fun fact: Brent Chapman is also known for creating the `majordomo` mailing list manager from the early 90s