Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The August 17 outage

github.blog

Post reply on HN
Print view View original

161–170 of 804 posts

Re: The August 17 outage

#161

Post by sqquima »

cube00 wrote: post #132

> Errors in those services triggered a client-side retry loop that increased traffic during recovery Symptomic of a wider trend to avoid showing the user any error at all costs, even if that means they sit watching a spinner for 7 hours. > Delayed replies to a single internal endpoint triggered a latent retry bug in VS Code that amplified traffic by approximately 10x and caused delayed recovery for the Copilot Token…

Maybe the retry logic was vibecoded instead of using an existing hardened library. After all, according to Twitter, nobody is looking at the code anymore.

Re: The August 17 outage

#162

Post by john_strinlai »

nycpig wrote: post #9

Almost 8 hours of downtime across all core workflows, and the word "sorry" or "apologize" appears nowhere in this post. "If you were trying to ship software that day, we let you down" is classic corporate non-apology speak. I’m done.

i am confident that if "sorry" appeared, someone would make a comment about "hollow apologies" or similar.

Re: The August 17 outage

#163

Post by bragr »

saghm wrote: post #141
xyzsparetimexyz wrote: post #52

Earlier quoted context omitted.

wow. they should really institute a maximum amount of individual pushes per-month per-user.

I still remember when they decided to limit the number of private repos you could have as a free user. Kind of silly to me at the time, and even more so now!

I remember when you had to pay for private repos in the first place.

Re: The August 17 outage

#164

Post by seizethecheese »

Aachen wrote: post #87
seizethecheese wrote: post #81

Earlier quoted context omitted.

Impressive for whom? It's impressive for the service to have such growth at that scale, the code being slop is somewhat irrelevant. Your comment just seems like mood affiliation (AI should be dismissed, growth was from AI, therefore growth should be dismissed).

How is it impressive if we all know it's autogenerated? There's no more people there than there were before. Heck, at ~2x growth that's possibly a decrease in real humans there since bots generate loads of them per person

> there's no more people there than there were before

Seems false. Lots of coding adjacent people, engineering managers, etc. are now pushing PRs.

> How is it impressive if we all know it's autogenerated?

Nobody is saying the code is impressive, just the growth of github is impressive. It's not doing anythign different based on the source of the code.

Re: The August 17 outage

#165

Post by wegwerper »

toephu2 wrote: post #77
blakesterz wrote: post #4

"Since April, monthly commits have grown from 1.4 billion to 2.9 billion. " Wow, that is some incredible growth in a really short time.

Not that impressive when you realize it's mostly due to AI slop edit: AI actually writes 99.9% of my code these days. I'm just saying of course the number of commits to github is going to climb astronomically due to AI.

Agree - this reminds me very much of the old joke of two economists increasing GDP by taking turns giving the same 200usd back and forth for having each other eat shit.

Useful / impressive for whom is the question. Not for us!

We pay for Github enterprise, and because GH can't be bothered to separate service tiers for sloplords and actual paying customers we get garbage level performance. They could of course always implement usage limits, but the goal is not to earn money, or provide a good service, the goal is to maximize AI users. Would be very awkward at the next executive golf meetup if you couldn't point to increased AI adoption.

In short: This is why monopoly laws matter. Once a company becomes too large, normal business rationales cease to be the motivation for their actions, and GH can go along with the pied piper of AI psychotic C-suite officers like MS is doing instead.

Re: The August 17 outage

#166

Post by riffraff »

saghm wrote: post #141
xyzsparetimexyz wrote: post #52

Earlier quoted context omitted.

wow. they should really institute a maximum amount of individual pushes per-month per-user.

I still remember when they decided to limit the number of private repos you could have as a free user. Kind of silly to me at the time, and even more so now!

Initially there were no private repos for free users at all, it was a selling point for bitbucket that they offered that.

Re: The August 17 outage

#168

Post by Brian_K_White »

recursive wrote: post #148
tremon wrote: post #145

Earlier quoted context omitted.

Why is Github talking about number of commits here, and not pushes? Are there a lot of tools/people using github as an online editing platform?

I'm not following your line of questioning. Without ever using github as an online editing platform, you can do one push with two new commits.

Commits are not expensive, pushes are. You can do any number of commits before you do one push, unless you are editing online, in which case every act is it's own commit & push.

You can rig up a local ide to pathologically commit+push per save, but you can do literally anything, so what you can do is immaterial.

Re: The August 17 outage

#169

Post by jdm2212 »

cube00 wrote: post #132

> Errors in those services triggered a client-side retry loop that increased traffic during recovery Symptomic of a wider trend to avoid showing the user any error at all costs, even if that means they sit watching a spinner for 7 hours. > Delayed replies to a single internal endpoint triggered a latent retry bug in VS Code that amplified traffic by approximately 10x and caused delayed recovery for the Copilot Token…

A common pattern in highly available services is that sometimes you should retry immediately (because the node you hit is rolling/broken/overloaded, but the others aren't) and other times you should back off aggressively (because the service is degraded).

If your server indicates with 100% accuracy when to retry immediately vs backoff, AND if all your clients consume that information with 100% accuracy, things go great. But there are lots of situations where one or both of those breaks down.

Re: The August 17 outage

#170

Post by doginasuit »

dpweb wrote: post #41

Sorry to suggest this but if they charged everyone say $1/mo. it would absolutely help the massive surge from AI coding they seem to have had. I don't like paying for free stuff but gh certainly worth it.

You are always paying one way or another, I prefer to pay in dollars and not frustration, attention, or privacy.
Post reply on HN