Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident with Actions and Pages

githubstatus.com

Post reply on HN
Print view View original

281–290 of 407 posts

Re: Incident with Actions and Pages

#281

Post by registeredcorn »

jaapz wrote: post #62
drcongo wrote: post #55

Earlier quoted context omitted.

Same. It's weird how I always find out that GitHub is down before GitHub does. Took 15 minutes before it appeared on githubstatus.com

All these monitoring rules are of the format "when 500 errors > baseline for x minutes". Otherwise you'd have monitoring alerts every second. So it is normal for users to already see errors before github officially counts it as an outage.

I'm not arguing with what you're saying, but it does make me wonder: What exactly is the point of the status page, if "it is normal for users to already see errors before GitHub officially counts it as an outage"?

Is it more so to have something to link to for managers who aren't using the service have a pretty bar to look at and feel like they are "doing something"? Or is it more of a kind of a way to prevent confirming what you already suspect to be true. E.g. "Huh. Me and Jim are seeing problems. How about you Tom? Oh wait, crud. The service page is confirming it's down now. Never mind! Who wants coffee?!"

Re: Incident with Actions and Pages

#282

Post by onion2k »

neya wrote: post #255
a10c wrote: post #19

My action failed with "Unexpected error fetching GitHub release for tag refs/heads/master: HttpError: Sorry. Your account was suspended" Which certainly made me shit myself, briefly.

It's an eye opener. Think about it - today, it was a mistake. But, what if it really happened? What if you really lost access to all your years of hard work? It's a wake up call. A blessing in disguise to store what matters to you the most locally, backed up offline. Never trust any single provider. Be it MS or Google or Apple. RAID is the way.

People should use something that keeps a local copy of their code and just copies it to Github and to other contributors with a sync process to push and pull changes. Some sort of 'distributed source control system' maybe. Then people would only need a 'hub' to connect to people, and it'd be easier to move somewhere else.

Re: Incident with Actions and Pages

#283

Post by 12_throw_away »

ModernMech wrote: post #187
insanitybit wrote: post #25

Earlier quoted context omitted.

It's (a) they're under massively increased load because everyone's vibing up new projects these days, (b) they've been in a weird frankenstein "on azure but also we have our own control plane" state for years and they're pushing to no longer have that be the case. I don't think vibecoding at Github has much to do with it.

I started using an agent (Codex) on my repo and it went from a a few dozen clones to thousands (3383 this week). I dunno what the agents are doing to clone the repo so many times -- I'm not running 3000 agents or prompts, maybe 10 or so this week. But if this is typical, a 1000x increase in usage across the board can't be good on the system.

> I dunno what the agents are doing to clone the repo so many times

agentic "ai" is going great

Re: Incident with Actions and Pages

#284

Post by 999900000999 »

sleight42 wrote: post #269
999900000999 wrote: post #230

Earlier quoted context omitted.

Ok. Move to EC2. Darn AWS is down. Alright, run it on a Mac Mini in your basement. Ahh dawn, your ISP is having issues. Good thing you have a backup 5G hotspot. Ohh no, the power is out. Eventually you have to trust someone else. GitHub is a tragedy of the Commons. Too many people are using it, and Microsoft isn't willing to handle it correctly. Feels like a very good business opportunity. Minimum 50k yearly contract…

It's almost as though GitHub should never have let itself be sold to Microsoft...

I'm sure the VCs who invested in GitHub disagree.

This is supposed to be Hacker News! Who is coming up with a startup to fill the gap !

Re: Incident with Actions and Pages

#285

Post by matt_kantor »

bouk wrote: post #271
cryo32 wrote: post #208

Earlier quoted context omitted.

You should never entirely depend on a third party service for deployments. Been burned too many times on that one.

We could still deploy manually but it's suboptimal! And we're 'flying blind' without CI runs

> And we're 'flying blind' without CI runs

You should never entirely depend on a third party service to run your tests, either.

Re: Incident with Actions and Pages

#286

Post by gbear605 »

13hunteo wrote: post #262
surdu wrote: post #257

Earlier quoted context omitted.

It was pretty easy before October 2018, when Microsoft bought them: https://www.githubstatus.com/uptime?page=31

Scaling to human use vs automated/agentic use is a very different thing.

Github has had horrible uptime for years, before agentic use was a thing. The killer was Microsoft.

Re: Incident with Actions and Pages

#287

Post by corvad »

neya wrote: post #255
a10c wrote: post #19

My action failed with "Unexpected error fetching GitHub release for tag refs/heads/master: HttpError: Sorry. Your account was suspended" Which certainly made me shit myself, briefly.

It's an eye opener. Think about it - today, it was a mistake. But, what if it really happened? What if you really lost access to all your years of hard work? It's a wake up call. A blessing in disguise to store what matters to you the most locally, backed up offline. Never trust any single provider. Be it MS or Google or Apple. RAID is the way.

RAID is not a backup.

Re: Incident with Actions and Pages

#289

Post by discordianfish »

surdu wrote: post #257
discordianfish wrote: post #247

Earlier quoted context omitted.

Because scaling complex systems is not trivial

It was pretty easy before October 2018, when Microsoft bought them: https://www.githubstatus.com/uptime?page=31

Not saying they are doing a particular good job but its not as simple as "paying more for bigger machines" and be done with it.
Post reply on HN