Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

Post reply on HN
Print view View original

771–780 of 1001 posts

Re: Incident with Github.com [resolved]

#771

Post by adamddev1 »

ALLTaken wrote: post #753
adamddev1 wrote: post #697

Earlier quoted context omitted.

I was in a very similar situation and this is what I did: - put Forgejo on a little $5 Vultr VPS (I share the VPS with other things, Forgejo doesn't need much resources. I also added some block storage to host my Docker containers in the Forgejo Docker registry) - got a 8GB ram cheaper Hetzner VPS (you have to wait for their availability) for around 8 EUR and used that to install the CI runner - Use Netlify for hosti…

Great setup, two questions, do you have 2 VPNs now or all on one Hetzner VPS? And how do you setup security? Just recommend swapping vercel/netlify with something else, we had quite a few issues and high costs with them. Not opiniated here, currently using Cloudflare and BunnyCDN (which is great)

I just SSH into the servers when I need to. I don't have any VPNs with these servers. I just try to follow normal server security setup practices, disabling password authentication etc.

Re: Incident with Github.com [resolved]

#772

Post by forgejojoj »

spockz wrote: post #741
sznio wrote: post #204

Earlier quoted context omitted.

my personal forgejo instance has had 100% availability for the last two years.

So it has run on a server that hasn’t had updates in 2 years? And it itself hasn’t been updated in 2 years?

The key is that maintenance is done on your own terms, and thus it's available when you need it.

Re: Incident with Github.com [resolved]

#773

Post by __float »

Aeolun wrote: post #727
jubilanti wrote: post #637

Almost 3 hours in and still "We are still working to identify the root cause" Can't even view diffs in the web interface! I'm out! I am willing to pay ~$5-10/mo for a reliable host I can swap as painlessly as possible. Mostly just building little static sites and little PWAs, no more than 500mb a repo and 10gb total. Need a repo with PRs and issues, where people can register free account and contribute or raise issue…

I’m working on it. Unfortunately ‘high reliability’ also means ‘tons of testing’, and ‘finding all the performance holes before your customers’. It’s kind of a shame, because Github keeps giving me fantastic ways to toot my own horn, but I can hardly do that if I can’t improve on their numbers.

Architecturally, how are you storing Git data? If it isn't fundamentally different from bare-repo-on-disk with libgit2 or wrapping the git binary, you're not going to scale any better.

Re: Incident with Github.com [resolved]

#774

Post by dwedge »

Treegarden wrote: post #633
fallingbananna wrote: post #489

Earlier quoted context omitted.

Personally I never engaged in the community aspect of GitHub. In my eyes it would be a ginormous gain to our colective mental health to not have the expectation to contribute to open source projects for free, or to have "green squares" every day on a public GitHub profile.

This seems to be such a non issue to me, I always ignored this and its strange to hear others feel compelled.

Especially now with LLMs. My hand coded github activity is through the roof the last two years just because of a job change, but I assume everyone's activity is up in the last 2 years for other reasons and if it meant anything in the past it definitely doesn't now

Re: Incident with Github.com [resolved]

#775

Post by Aachen »

BowBun wrote: post #429
Aachen wrote: post #225

Earlier quoted context omitted.

Huh, they usually do? I'm not a full-time developer though so I'm surely biased towards doing the obvious quick fixes or already-discussed things, but discussing how to solve a problem before throwing something at the wall seems like the normal thing to do for me Or is their point about abandoned projects where the dev never clicks release after merging, or even never merges?

The joke was that we have open pull requests that we cannot merge because of this downtime.

Yes, see sibling comment posted slightly ahead of yours (dead/invisible at first, to be fair)

Re: Incident with Github.com [resolved]

#776

Post by regexorcist »

thanh76 wrote: post #683
khvn26 wrote: post #29

I had a lot of goodwill for GitHub but I think today is the tipping point. Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again. The hope is dead.

I think the same. But the GitHub CLI is the biggest block for me. It is a killer feature. No other tool is close. To self-host the forge is easy. To replace gh in my scripts and agent workflows is not.

Any good LLM should be able to come up with the equivalent git and curl commands fairly easily.

Re: Incident with Github.com [resolved]

#777

Post by TwelveEyes »

datsci_est_2015 wrote: post #258
askonomm wrote: post #139

Earlier quoted context omitted.

That's an overly optimistic view of things. For a lot of us GitHub is critical infrastructure, which if it goes down loses us and our customers money.

It is interesting, how much money is being lost during this outage? My significant other was just let go from their job as a scapegoat for an organizational error: 3 layers of failure - IC, manager, director, and the IC was let go. The error caused a 7 figure loss for the company that has 10 figures of revenue per year. The manager and director may not see any consequences, though the director will probably be forced…

Is responding calmly a privilege to have, or a kindness to yourself?

Re: Incident with Github.com [resolved]

#778

Post by [deleted] »

phtrivier wrote: post #212
leishman wrote: post #171

I don't understand why Github hasn't solved this problem with pricing updates. My understanding is they are getting hammered with LLM generated code growing their traffic by over an order of magnitude. So why not rate limit non-paying users and charge for whatever scarce resource is being consumed that is causing them to constantly fall-over? This seems like a basic economics problem.

They also sell the machinery to produce the slop (through OpenAI at least). If they make it harder to push slop on github, people might treat it as an incentive to generate less slop. Less tokens sold. Bad. Very bad. The github paying customers are not going to move away. Plus, it's probably genuinely hard to scale. And they can't throw compute at the problem - all compute is already going to the LLMs.

[deleted]

Re: Incident with Github.com [resolved]

#779

Post by internetter »

BryanBeshore wrote: post #242
leishman wrote: post #171

I don't understand why Github hasn't solved this problem with pricing updates. My understanding is they are getting hammered with LLM generated code growing their traffic by over an order of magnitude. So why not rate limit non-paying users and charge for whatever scarce resource is being consumed that is causing them to constantly fall-over? This seems like a basic economics problem.

They would benefit from a HackerNews-esque karma system.

That is essentially your number of followers. Pretty hard to have a system that cannot be gamed. I think followers are probably loosely correlated with tech ecosystem impact in some capacity, but the second there is an incentive to accumulate them, well...

Re: Incident with Github.com [resolved]

#780

Post by SkySkimmer »

s_dev wrote: post #145

I recall reading years ago that cloud services were expected to run with a reliability of 3 or 4 '9's and that if they didn't competing services would quickly overtake them in adoption. The industry was supposed to be that cut throat. Has big tech reached a similar status like banks in that they are "too big to fail" i.e. when they do fail we all just look the other way and say: "well everyone else is out too". Didn'…

Github is a social network, not a cloud service.

(only 5% /s)

Post reply on HN