Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why are most status pages delayed?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 62 posts

Ask HN: Why are most status pages delayed?

#1

Post by 2gremlin181 »

As I type this, Reddit is down. My own requests return 500s, Down Detector reports that there is an outage but Reddit itself says all systems operational.

This is a pattern that I have noticed time and time again with many services. Why even have a status page if it is not going to be accurate in real time? It's also not uncommon that smaller issues never get acknowledged.

Is this a factor of how Atlassian Statuspage works?

Edit: Redditstatus finally acknowledged the issue as of 04:27 PST, a good 20+ minutes after Down Detector charts show the spike

Re: Ask HN: Why are most status pages delayed?

#2

Post by Bender »

Bureaucracy. Companies have service level agreements with other companies. They want to be damn sure they can not disavow an outage before something says there is an outage. There will in most cases be a process involved in updating the status page that will intentionally have many layers of bureaucracy hurdles to jump through including many approvals. The preference will often be to downgrade an "outage" to a "degradation" or "partial outage" or some other term to downplay it and avoid having to pay credits on their B2B service level agreements and such.

Re: Ask HN: Why are most status pages delayed?

#4

Post by bithaze »

> Why even have a status page if it is not going to be accurate in real time?

The funny thing is reddit's status page used to have real-time graphs of things like error rate, comment backlog, visits, etc. Not with any numbers on the Y-axis, so you could only see relative changes, really, but they were still helpful to see changes before humans got around to updating the status page.

Re: Ask HN: Why are most status pages delayed?

#5

Post by knorker »

To add to the reasons others gave: It needs to be correct.

Engineers are working the problem. They have a pretty good understanding of the impact of the outage. Then an external comms person asks for an engineer to proof read the external outage comms. Which triggers rounds of "no, this part is not technically correct" and "I know the internal system scope impact, but not how that maps to external product names you want to communicate".

Sure, it'd be nice if the message "we are investigating an issue with… uh… some products" would come up faster.

Re: Ask HN: Why are most status pages delayed?

#8

Post by bberenberg »

Alert fatigue. Down Detector will show an outage with a service when the intermediate network is down. Companies have to triage alerts and once they’re validated they are posted on a status page. Some companies abuse this to hide their outages. Others delay in a reasonable manner.

I have considered building something to address this and even own honeststatuspage.com to eventually host it on. But it’s a complex problem without an obviously correct answer.

Re: Ask HN: Why are most status pages delayed?

#9

Post by 2gremlin181 »

knorker wrote: post #5

To add to the reasons others gave: It needs to be correct. Engineers are working the problem. They have a pretty good understanding of the impact of the outage. Then an external comms person asks for an engineer to proof read the external outage comms. Which triggers rounds of "no, this part is not technically correct" and "I know the internal system scope impact, but not how that maps to external product names you w…

I totally get that, but how hard would it be to actually make calls to your own API from the status page? If it fails, display a vague message saying there might be issues and that you are looking into it. Clearly these metrics and alerts exist internally too.

I'm not asking for an instant RCA or confirmation of the scope of the outage. Just stop gaslighting me.

Re: Ask HN: Why are most status pages delayed?

#10

Post by giancarlostoro »

bberenberg wrote: post #8

Alert fatigue. Down Detector will show an outage with a service when the intermediate network is down. Companies have to triage alerts and once they’re validated they are posted on a status page. Some companies abuse this to hide their outages. Others delay in a reasonable manner. I have considered building something to address this and even own honeststatuspage.com to eventually host it on. But it’s a complex proble…

I've seen all sorts of Azure outages that never wind up on their status page. Granted they could be unique to a small pool of services.
Post reply on HN