Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

Post reply on HN
Print view View original

811–820 of 1001 posts

Re: Incident with Github.com [resolved]

#811

Post by cortesoft »

adamddev1 wrote: post #697
jubilanti wrote: post #637

Almost 3 hours in and still "We are still working to identify the root cause" Can't even view diffs in the web interface! I'm out! I am willing to pay ~$5-10/mo for a reliable host I can swap as painlessly as possible. Mostly just building little static sites and little PWAs, no more than 500mb a repo and 10gb total. Need a repo with PRs and issues, where people can register free account and contribute or raise issue…

I was in a very similar situation and this is what I did: - put Forgejo on a little $5 Vultr VPS (I share the VPS with other things, Forgejo doesn't need much resources. I also added some block storage to host my Docker containers in the Forgejo Docker registry) - got a 8GB ram cheaper Hetzner VPS (you have to wait for their availability) for around 8 EUR and used that to install the CI runner - Use Netlify for hosti…

This sounds great for personal projects, but what about open source work? Or if you want to collaborate with more than a few people?

Re: Incident with Github.com [resolved]

#812

Post by intrasight »

ajmurmann wrote: post #758
rurban wrote: post #700

Earlier quoted context omitted.

My blackest Monday today. Deepseek is off/too expensive, Claude tokens out for 3 days, kimi refuses, and Github down for the whole afternoon. Maybe I'll take the week off

It's crazy that it wasn't that long ago that all we worked with was a compiler on the local machine and maybe a SVN server you'd interact with every few hours

Not that long ago that I used a local CVS server for SCCS. OK, almost 40 years - but still.

Re: Incident with Github.com [resolved]

#813

Post by palata »

jillesvangurp wrote: post #717
palata wrote: post #527

Earlier quoted context omitted.

> throw out the baby with the bathwater by focusing on self hosting without attempt at networking with other instances May I ask what networking with other instances brings? Git is decentralised, so it works with self-hosted instances, right?

Git is decentralized in the sense that you can clone your repository wherever. But there is no real networked community on top. The first versions actually piggy backed on the social network that is email. Linux still is managed via email. It's social network was mailing lists. By decentralizing, they could avoid the pesky topic of having to give each other read write access to each other's git servers. But if you wa…

> That makes it the best of both worlds

Assuming that everybody moves to your community (in this case the atproto one). Then the ActivityPub people ask you to join their community, and so does every new community that wants to become the new standard.

That's the thing with every new social network: "You don't want to be locked into the current community that is not my community, instead you want to move to my community, which is the best! And if you do, and convince everyone around you to join as well, then you will only ever need one account for everything!".

And now, assuming that most people leave GitHub and join Tangled, it will be to the detriment of the minority that is not using Tangled. If I host a repo on SourceHut, nobody will contribute because they want GitHub. If Tangled replaces GitHub, will that change? I don't think so, people will just refuse to use anything other than Tangled.

What I find interesting is that you advocate for a federated system that everyone should use, because "it would be better for everybody", but the fact is that everybody has an email already. No need to create an account, we can send each other patches, it just works.

Why don't you promote email? Because it's not your preference.

PS: I do realise that email has a lot of issues. My point is just that I avoid the entire issue by just being open to sharing my code however it works. And I feel like patches are easier to share "universally" than pull requests. Maybe I would be in favour of a new, "richer" patch format that would make it even nicer to share over whatever system I want (email, atproto, activitypub, signal, ...)?

Re: Incident with Github.com [resolved]

#814

Post by red-iron-pine »

datsci_est_2015 wrote: post #744
pkilgore wrote: post #735

Earlier quoted context omitted.

Name names.

Local (to us) firm. Two main reasons I didn't: - Have never seen it mentioned on this forum, in any context. - We're potentially pursuing legal action.

names or it didn't happen, mate

50% of social media traffic is bots and i want to believe

Re: Incident with Github.com [resolved]

#815

Post by ToucanLoucan »

ALLTaken wrote: post #766
nxc18 wrote: post #500

Earlier quoted context omitted.

The quality and performance of GitHub enterprise has degraded horribly this year. Their problems are self-inflicted and influx is just an excuse. Despite minimal new features, and drastically increasing hardware resources to keep up, self-hosted GitHub enterprise went from things being instant to most requests taking many seconds. And activity on issues in one repository tends to mysteriously re-open issues with the…

Please let this not be true :( I mean nothing against vibe coding for hobby and assisting you when you get stuck, but at something important like Github they should have hooks to stop low-quality commits and repeated CI-breaking or issue causing should automatically restrict accounts at such companies I think. Do you think they have such measures in place? I remember when my indention wasn't spotless our university r…

It’s become clear to me that any company acquired by Microsoft should be avoided. All software dev is hard. Software dev within a huge organization is damned hard. And Microsoft is SO HUGE and has so much infighting and finger pointing going on that they’re now, as far as I’m concerned, incapable (let alone unwilling) to ship good software.

No faith left. None. They turn everything into a quagmire.

Re: Incident with Github.com [resolved]

#816

Post by red-iron-pine »

faeyanpiraat wrote: post #308
antaviana wrote: post #207

Earlier quoted context omitted.

"It's just money. It's made up. Pieces of paper with pictures on it so we don't have to kill each other just to get something to eat". Jeremy Irons in movie Margin Call

You cant eat a quote

but you can download a car

Re: Incident with Github.com [resolved]

#817

Post by maccard »

mitchjj wrote: post #418
beardedetim wrote: post #15

We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?

Worth being upfront with, I'm an employee, but; Buildkite. Frontier labs and global saas heavyweights run CI through us, plus a crop of open source tooling like Bazel, vLLM, Firecracker, Rails, Ray, Sorbet and Rust. Shopify, Uber, Airbnb, Canva stresstested the platform ten years ago for scaling issues. You need to take a little time to design upfront and take advantage of dynamically programmable primitives. It isn'…

I want to second build kite - it’s an excellent piece of tech and is best described as “sane Jenkins” as a compliment. My only gripe is the pricing - it’s not designed for “small teams”. But I’d use it again in a heartbeat.

Re: Incident with Github.com [resolved]

#818

Post by brw »

1matin wrote: post #716
varun_ch wrote: post #644

Earlier quoted context omitted.

What are the serious holes?

The main hole is that ATProto's data model is a bit awkward for a code forge. Issues/PRs/comments are records owned by their authors, not by the repo. So an issue can span N users' PDSes. That's a neat model for social media, but a forge wants the repo to be the durable owner/context of its issues and history because that's where the comment/issue/etc. logically belongs. The solution? tangled.org acts as a super adva…

There's a good chance this will change in the future and the repo on the knot will also contain the social aspects of repos (issues, PRs, etc.)

> As time goes on we are re-assessing the idea of users owning what is "collaborative data" (issues, PRs, etc.) on their PDSes - soon may come the day that an issue also lives on the knot as a source of truth, with an accompanying pointer record on user PDS to attest that it's theirs.

from https://tangled.org/tangled.org/core/tree/master/knot2

Re: Incident with Github.com [resolved]

#819

Post by 19skitsch »

tom1337 wrote: post #677
jubilanti wrote: post #637

Almost 3 hours in and still "We are still working to identify the root cause" Can't even view diffs in the web interface! I'm out! I am willing to pay ~$5-10/mo for a reliable host I can swap as painlessly as possible. Mostly just building little static sites and little PWAs, no more than 500mb a repo and 10gb total. Need a repo with PRs and issues, where people can register free account and contribute or raise issue…

Update 16:36 UTC - We identified the problematic component and have taken corrective actions. There are strong signs of recovery but we are still working to completely restore service, with error rates still remaining slightly elevated. We will post further updates as recovery continues. just shy of 3 hours to find the issue

I mean to be fair it’s a huge production system, there are many moving parts. Having worked on outages before in complex systems it is very difficult to identify root causes under pressure

Re: Incident with Github.com [resolved]

#820

Post by red-iron-pine »

parthdesai wrote: post #411
dewey wrote: post #363

Earlier quoted context omitted.

If you pay developers x money / day and one of their core tools is down for n hours during the day and they spend their money on HN instead that's pretty straight forward to calculate.

I meant more for their customers specifically

we push customer code on mondays. it is monday. code is not being pushed.

we do not bill.

the outages they promised their customers are now different.

this has costs for everyone downstream.

Post reply on HN