GitHub was down • Hacker News Board

Re: GitHub was down

#281

Post by COMMENT___ » Thu, Aug 15, 2024, 6:30 PM UTC

xp84 wrote: post #169

cbates wrote: post #102

Earlier quoted context omitted.

Seems slightly unproffesional for a massive company like Github/Microsoft.

I disagree. This hurts no one, and not everything needs to be sanitized and painted over with bland corporatespeak.

This “ironic” and “humorous” style of errors and UI captions is the actual new corporate speak. I’d prefer dumb error messages rather than some shit someone over the ocean thinks is smart and humorous. And it’s not funny at all when it’s a global outage impacting my business and my $$$.

Re: GitHub was down

#282

Post by arjvik » Thu, Aug 15, 2024, 7:17 PM UTC

erksa wrote: post #179

The mobile app on iOS is a 503 with ``` Received a 503 error. Data returned as a String was: Hello future GitHubber! I bet you're here to remove those nasty inline styles, DRY up these templates and make 'em nice and re-usable, right? Please, don't. https://github.co ... ``` That's where it's cut off on my screen. Curious what the link is :) I like to think, someone did.

Anyone seen the full text of the error page?

Re: GitHub was down

#283

Post by blt » Thu, Aug 15, 2024, 9:06 PM UTC

raxxorraxor wrote: post #261

blt wrote: post #255

Earlier quoted context omitted.

Making logins required to view twitter was the ultimate bed shitting move. The whole point of twitter was to be a broadcast medium. Tweets were viewable without following or logging in. There is a huge vacuum in that space now.

For most (social media) platforms really. Management believes it would force users to sign up, but in reality the platform just becomes less relevant because of that limitation. Not even talking about search crawlers. An all around stupid decision. That said, if management is that shitty, the platform probably won't be attractive for long anyway. Facebook/Instagram were successful despite that to a degree, but this d…

I don't really agree with that. Facebook was originally about mirroring your real-life social network. In the mid 00's nobody was trying to get likes from strangers on Facebook.

Instagram is closer to broadcast, but it was always closely tied to the mobile app experience and the "follower" mentality. People didn't really share links to Instagram posts in other online venues in the beginning.

Twitter was always unique. It existed before smartphones, and there was a good chunk of years where people without smartphones would read twitter posts on desktops. Its producer/consumer distribution is much more skewed, many twitter users never post. Tweets were always getting posted to places like HN, reddit, discussed in news articles, etc.

I think Twitter's (former) position as a broadcast medium à la TV, radio, and newspapers is unique among social networks. There's a reason why Twitter was the place for journalists, politicians, academics, fire departments, web service status alerts, etc.

Re: GitHub was down

#284

Post by rozenmd » Thu, Aug 15, 2024, 9:23 PM UTC

sebmellen wrote: post #7

I've never seen an outage this big. Even the homepage doesn't load. We've had recurrent issues with Actions not running, but this seems a lot bigger. The status page says all is well, though: https://www.githubstatus.com/ . Hilarious.

Here, we caught 35 minutes of downtime: https://github.onlineornot.com/incidents/6Yyj8YWD94zE

Re: GitHub was down

#285

Post by bamboozled » Fri, Aug 16, 2024, 2:05 AM UTC

kgrax01 wrote: post #205

bamboozled wrote: post #202

Earlier quoted context omitted.

How would customer credentials being leaked be part of an outage of this size ?

If its enough of a security issue they could have pulled the site while its fixed/cleaned

If that was the case, you'd think it would've been a less dramatic outage right? Maybe even a status page to indicate that?

Re: GitHub was down

#286

Post by t_believ-er873 » Mon, Aug 19, 2024, 12:38 PM UTC

Unfortunately, outages happen... This situation is a very good reminder of why having backups and a solid Disaster Recovery plan is crucial. Of course, it’s easy to assume that cloud services are always up, but we should never forget about outages. Setting up automated backups for repos and metadata can save a ton of headaches when things go wrong. Plus, having a Disaster Recovery plan means you’re not stuck waiting for the service to come back online—you can keep working with minimal disruption.

Here is a good article on how to prepare for the situations like that, when GitHub is down: https://gitprotect.io/blog/github-restore-and-github-disaste...

Re: GitHub was down

#287

Post by miloszgp2000 » Tue, Aug 20, 2024, 4:12 PM UTC

This was not the first or last GitHub outage. Unfortunately, this was a big one. That why now it is even more important to have backups of your work and restore capabilities in case of a scenario like this outage. This article sheds light on the importance of backups along with the best practices to follow:

https://gitprotect.io/blog/github-backup-best-practices/

Re: GitHub was down

#288

Post by fragmede » Thu, Aug 22, 2024, 5:59 AM UTC

laserlight wrote: post #258

fragmede wrote: post #250

Earlier quoted context omitted.

Where on the continent? GitHub is undoubtedly doing blackbox testing internally and has multiple such monitors but that's not going to capture every customer's route to them, leading to the same problem - customers experience GitHub being down, despite monitoring saying it's mostly up. Thus the impass. Even doing whitebox testing, where you know the internals and can this place sensors intelligently, even just for in…

> 100% fidelity is impossible I don't think anybody asked for 100% fidelity. We are talking about a complete outage that affected at least North America and Europe. If the status page shows green in such a case, its fidelity is around 50%. People expect better from GitHub.

The amount of moaning that the status page wasn't updated in 0 seconds and had the wrong status for entire minutes is what leads me to believe that no, users do expect 100a% fidelity.

Total outages are rare enough, and there's enough other work, that spending time building a system for that, just doesn't seem like the best use of their time. though I'm biased, having faced that exact question from the inside, at different company.