Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Is Down

githubstatus.com

Post reply on HN
Print view View original

61–70 of 72 posts

Re: GitHub Is Down

#61

Post by PapaPalpatine »

japhyr wrote: post #22

There's a not-entirely-small part of me that hopes Issues breaks entirely for long enough that people at GitHub rethink their implementation. GH issues have been so useful for the better part of a decade, that I have an empty repo called my_life just so I can make issues about things like home maintenance. In the last few months, the UI has become so slow and flaky that I don't use it for nearly as much as I used to.…

I’m low-key certain it’s because they started using more React on the front end, especially for the new Issue stuff. The entire UI feels slow and janky like GitLab.

Re: GitHub Is Down

#62

Post by joseda-hg »

japhyr wrote: post #22

There's a not-entirely-small part of me that hopes Issues breaks entirely for long enough that people at GitHub rethink their implementation. GH issues have been so useful for the better part of a decade, that I have an empty repo called my_life just so I can make issues about things like home maintenance. In the last few months, the UI has become so slow and flaky that I don't use it for nearly as much as I used to.…

For something so critical, does it make sense to self host say a gitea instance or such?

I know you can sync repos, I don't think you can sync issues tho

Re: GitHub Is Down

#63

Post by flyinghamster »

tovej wrote: post #13

It's not just PRs and issues, my build systems are breaking because I can't download tarballs for dependencies from github.

One thing I started doing a few years ago is that, any time I check out a git repo, whether it's on github or anywhere else, is clone it to my NAS and then clone from there to whatever I'm building it on. Aside from guarding against losing dependencies to outages (or from being yanked), it can be handy for guarding against my own fat fingers, and for making sure that the same version is running on multiple systems.

Re: GitHub Is Down

#64

Post by lawrjone »

diggan wrote: post #33
lawrjone wrote: post #24

The industry made a general decision after ZIRP ended to deprioritise availability after years of historic levels of engineering investment. It's no surprise we're now feeling those effects but damn, GitHub and other services like Slack have been really bad lately.

Or, GitHub used to favor stability back in the day, and there wasn't a lot of changes. People were complaining that GitHub didn't "improve" enough day-to-day, so after the Microsoft purchase, Microsoft started forcing GitHub to add more features, stability be damned. Most outages are caused not by stuff just randomly breaking, but updates/upgrades going wrong. If you try to increase the output of new features/changes…

I'm not totally certain you can attribute it to this. I know there's a bunch of work going on to migrate things into Azure after the MS acquisition but it feels like more of an industry trend that we cut engineering spend at the cost (often) of lower quality output and outages like these.

GitHub laid off 10% of their staff in 2023 and like you say, won't have slowed down to account for that.

Ignoring all that though... other than Copilot, what big feature changes have you seen in GitHub? My experience of using their product has been broadly unchanged for years.

Re: GitHub Is Down

#65

Post by G1N »

They're saying PRs and issues but I can't access github.com or github oauth for logging into external services. Need to migrate our org off of using github auth for anything critical I think, this is twice so far this year that we've had these issues.

Re: GitHub Is Down

#66

Post by lawrjone »

NewJazz wrote: post #56
lawrjone wrote: post #24

The industry made a general decision after ZIRP ended to deprioritise availability after years of historic levels of engineering investment. It's no surprise we're now feeling those effects but damn, GitHub and other services like Slack have been really bad lately.

Citation needed? Is this true beyond github and slack? Both were acquired btw, that could explain their availability issues (just like twitter, now X). What about e.g. AWS or GCP? Has their availability meaningfully reduced?

I wouldn't say this applies to cloud providers, they have a very different business on their hands.

But for SaaS in general I think the trend is noticeable? Twitter led the way with massive layoffs in engineering often in the roles around reliability. The industry as a whole have aimed to cut costs however possible, and reliability/ops is usually seen as a cost-centre that gets hit hard.

I've watched this in my own space (start/scale-ups and larger companies, I work in incident response tooling) as people start talking very differently about reliability and engineering investment. You hear "do more with less" about five times every day and spend that was previously greenlit by default around reliability/redundancy is under much more scrutiny now.

I see this as a silent mirror of the reduction in open-source efforts from companies now there's been a refocus on business impact and bottom-line.

Re: GitHub Is Down

#67

Post by snapplebobapple »

joseda-hg wrote: post #62
japhyr wrote: post #22

There's a not-entirely-small part of me that hopes Issues breaks entirely for long enough that people at GitHub rethink their implementation. GH issues have been so useful for the better part of a decade, that I have an empty repo called my_life just so I can make issues about things like home maintenance. In the last few months, the UI has become so slow and flaky that I don't use it for nearly as much as I used to.…

For something so critical, does it make sense to self host say a gitea instance or such? I know you can sync repos, I don't think you can sync issues tho

this is what i doits worth the 11 dollars a year for the vps to control my own

Re: GitHub Is Down

#68

Post by diggan »

lawrjone wrote: post #64
diggan wrote: post #33

Earlier quoted context omitted.

Or, GitHub used to favor stability back in the day, and there wasn't a lot of changes. People were complaining that GitHub didn't "improve" enough day-to-day, so after the Microsoft purchase, Microsoft started forcing GitHub to add more features, stability be damned. Most outages are caused not by stuff just randomly breaking, but updates/upgrades going wrong. If you try to increase the output of new features/changes…

I'm not totally certain you can attribute it to this. I know there's a bunch of work going on to migrate things into Azure after the MS acquisition but it feels like more of an industry trend that we cut engineering spend at the cost (often) of lower quality output and outages like these. GitHub laid off 10% of their staff in 2023 and like you say, won't have slowed down to account for that. Ignoring all that though.…

> My experience of using their product has been broadly unchanged for years.

Compared to how fast/slow they were moving 2012-2018, they're moving at blazing speed now. It feels like every time I open GitHub now, there are new features/changes and "Beta" available stuff. The platform is almost completely different today than it was in 2018, for better or worse.

Re: GitHub Is Down

#70

Post by mrguyorama »

rvz wrote: post #36
bigfatfrock wrote: post #10

Who's going to kick off the holy war around self hosting?

It all began 5 years ago [0] when we wanted to 'centralize everything to GitHub.' Then, the outages accelerated from there. [0] https://news.ycombinator.com/item?id=22868406

"are you sure your self-hosted solution will have better uptime?"

Hilariously, after 15 years working in self hosted bitbucket systems, YES entirely.

An underfunded university with an incompetent but doing their best IT department? Zero downtime.

A mid sized company full of overly confident and "just build it" nerds building fragile shit? Zero downtime.

A large corporation with a completely outsourced IT department that can't give you access to something unless you do exactly the right undocumented thing in our internal ticketing software? Zero downtime.

That includes self hosted jenkins and literally homebuilt infrastructure with zero documentation and the guy that built it left a while ago.

I have never been able to blame our build and code infra for lack of productivity.

Post reply on HN