Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

721–730 of 1001 posts

Re: Incident with Github.com [resolved]

#721
post #29

I had a lot of goodwill for GitHub but I think today is the tipping point. Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again. The hope is dead.

I’m willing to give them a break as I’m assuming they have a lot of scaling problems due to the influx of LLM assisted coding. But maybe I’m wrong?

[flagged]

Re: Incident with Github.com [resolved]

#722

The most shocking thing to me about the entire GitHub fiasco is: No matter how bad the uptime is, no matter how bad the UI is. We will keep using the software / service we depend on. Why do we as software engineers have set fault tolerance and high availability as the primary goal of any software / infrastructure architecture? This shows it's completely irrelevant.

[deleted]

Re: Incident with Github.com [resolved]

#723
This weekend I managed to get a test into some sort of infinite loop that didn’t overflow or time out, so I had to single-step in to figure out where the last line it broke on. Something to do with events propagating. Oh and someone misunderstanding an unfamiliar mocking framework and not preserving someone else’s cleanup code. I won’t name and shame but his name stars with an h and ends with inkley. Mangled someone else’s abandoned PR thinking one cleanup function was a superset of another. It was not, dumdum.

Event systems have interesting ways to fail that resist throwing good engineering at them to fix. For a long time I refused to have anything to do with them, but now we have whole companies built on them and also things like React and what can you do? There are harmonic failure modes where the system doesn’t fail but an event keeps echoing forever and using up 50% of the available compute before anyone notices it’s been like that for months and slowly getting worse. And then there’s the Poison Pill where a message crashes a process, then gets picked up by the next and kills that one too until they’re all dead.

Re: Incident with Github.com [resolved]

#725

Almost 3 hours in and still "We are still working to identify the root cause" Can't even view diffs in the web interface! I'm out! I am willing to pay ~$5-10/mo for a reliable host I can swap as painlessly as possible. Mostly just building little static sites and little PWAs, no more than 500mb a repo and 10gb total. Need a repo with PRs and issues, where people can register free account and contribute or raise issue…

Sounds like they don’t have a fine enough lens on some particular class of problems. Been there before. The source of the meta problem is so old few people will learn the lesson that hadn’t known it already.

Re: Incident with Github.com [resolved]

#727

Almost 3 hours in and still "We are still working to identify the root cause" Can't even view diffs in the web interface! I'm out! I am willing to pay ~$5-10/mo for a reliable host I can swap as painlessly as possible. Mostly just building little static sites and little PWAs, no more than 500mb a repo and 10gb total. Need a repo with PRs and issues, where people can register free account and contribute or raise issue…

I’m working on it. Unfortunately ‘high reliability’ also means ‘tons of testing’, and ‘finding all the performance holes before your customers’.

It’s kind of a shame, because Github keeps giving me fantastic ways to toot my own horn, but I can hardly do that if I can’t improve on their numbers.

Re: Incident with Github.com [resolved]

#728

Go touch grass

I'll never get lack of sex used as an insult, unless you are a teenager. Is your self worth that tied to sexual activity so to use it as an insult? Is it that a rare thing to "achieve" for you to put it on a pedestal and demean others based on the lack of it?

Go touch grass isn't meant metaphorically. It literally means go outside. Has nothing to do with sex.

Re: Incident with Github.com [resolved]

#729

The most shocking thing to me about the entire GitHub fiasco is: No matter how bad the uptime is, no matter how bad the UI is. We will keep using the software / service we depend on. Why do we as software engineers have set fault tolerance and high availability as the primary goal of any software / infrastructure architecture? This shows it's completely irrelevant.

It is not. I am just not happy with the alternatives; Gitlab, and Codeberg, for well-discussed reasons. I will pounce as soon as the right one comes along, and I have a few on my radar.

Re: Incident with Github.com [resolved]

#730
post #671
post #228

Earlier quoted context omitted.

Services like Facebook have orders of magnitude more activity than GitHub and manage much better uptime.

Services like Facebook aggressively block automated activity, including AI agents. GitHub sells this as a service.

Bots or not, Facebook has orders of magnitude more traffic.
Post reply on HN