Live data from Hacker News

The August 17 outage

github.blog

541–550 of 804 posts

Re: The August 17 outage

#541
post #327

I applaud GitHub. However, I think no matter how valiant they are they will not climb out from under this. The scale problem will keep getting worse, and it's getting worse in a way I don't think is translating to more money for them. Sooner or later, they're going to have to charge for things currently free. I've been saying this for a while: https://news.ycombinator.com/item?id=47534499

If pushes are the main problem, they could rate limit too.

Re: The August 17 outage

#543
post #325
post #314

Earlier quoted context omitted.

Why don’t you have your actions run on pr or push instead of on commit? Why would you even want that? I’ve never seen actions set up that way. If I push a branch with 100 commits then it’ll only run CI once. It’ll show the rest of the commits in the UI, sure, but that doesn’t mean that it’s the same performance impact. It could very well be 1 db transaction with multiple rows written instead of 100. I think you’re re…

Github runs CI per PR push (obviously), and per main-branch commit (click on commit history in any project with CI, see a build result check mark on each one - that's true if you push a dozen commits too (I've done that)), in nearly all setups I've seen. I'm not sure how much of that is required vs default though. With enough effort, you can rather obviously run CI per PR commit (it's a programmable system), but I've…

Things like password scanning have to look per object uploaded

Re: The August 17 outage

#544

"Since April, monthly commits have grown from 1.4 billion to 2.9 billion. " Wow, that is some incredible growth in a really short time.

It's like cancer growth though, not the 'good' kind of growth ;) E.g. I doubt that the number of Github users has doubled in that month too. Github should probably introduce daily commit- and merge-limits that slow down excessive clanker activity, but are high enough that a human doesn't notice. Alternative put users with excessive resource usage on their own 'sub-infrastructure' so that when this is overloaded, the regular users are not affected by the outage.

Re: The August 17 outage

#545
post #414

Earlier quoted context omitted.

Bonkers is right. Where in those ~12 billion commits is the software, products and "innovations" which are supposed to be making our lives better? Software and apps in particular are getting worse, normies hate AI more than ever because they're even less likely to get their desired outcome when calling their doctor or trying to get their online order refunded when chatting with a cutely named chatbot, wages for (most…

This comment comes up over and over again and it's incredibly ignorant. To give just a single example, ai code dev has enabled people to make tools for themselves that they didn't have before. I've made a language learning app for myself. Its working better than Duolingo so far, for me. Its not really public

Speaking of ignorance, how does your success story relate to the societal issues GP brought up?

Re: The August 17 outage

#546

"Since April, monthly commits have grown from 1.4 billion to 2.9 billion. " Wow, that is some incredible growth in a really short time.

this is impressive too

"We have since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity"

Re: The August 17 outage

#547

"Since April, monthly commits have grown from 1.4 billion to 2.9 billion. " Wow, that is some incredible growth in a really short time.

LOL, this kind of things will happen when projects like Bun ( https://github.com/oven-sh/bun ) are running on auto. :)

Wow...

I am currently using bun, but may have to switch. I can't see how this can possibly turn out well in the long run...

Re: The August 17 outage

#548
post #508

Earlier quoted context omitted.

So much more stuff and growing- what it is actually useful for ? Are we getting actually more done than with previous volumes or is it just all wasted energy?

Has anyone noticed an increase in the quality, performance or capabilities of the software they use?

Only in software that is used to make other software. So we're all just patting ourselves on the back within our bubble.

What I have noticed an increase though is in demands and pressure to deliver.

Re: The August 17 outage

#549
post #479

Earlier quoted context omitted.

That's how it's sold, but have you heard about any major tech company that sends their worker home when they've achieved what they used to achieve pre-AI? EDIT: Instead of simply down-voting, you're welcome to name examples that proves me wrong ;)

This is anecdotal, but I know that a lot of my coworkers and coasting and putting up one AI generated PR per day which they've hardly even self reviewed.

In a good company that will come back to bite them next performance review.

So, if it doesn't then you learned something about your workplace (and it's not good).

Re: The August 17 outage

#550

Earlier quoted context omitted.

The whole point of exponential backoff is that the first retry can be quick.

The right answer is for the RPC framework to accurately communicate "try again on another node" vs "don't try again, just hard fail". When one end user request fans out to hundreds of backend requests (typical for microservices), you can't have each of those backend requests do its own exponential backoff. If they do it in parallel, they're a thundering herd, and if they do it in serial, the end user request will tim…

Thats why you only back off at the edges, (not internally, where you can spin up on demand) and why you should add jitter to the timing.
Post reply on HN