Live data from Hacker News

GitHub incident 2022-03-23

githubstatus.com

191–200 of 206 posts

Re: GitHub incident 2022-03-23

#191
post #138
post #118

Earlier quoted context omitted.

Just a bone to pick... new grad engineers in my US city started around 60-70k in 2018 when my college cohort graduated. Southern US...

Things have changed considerably over the last four years.

It still isn’t that high except for at a hand few of places and even then they’ll start you off less but give you a total comp that exceeds. Still it isn’t far of the mark (at least in Seattle where I live.)

Re: GitHub incident 2022-03-23

#192

I'm inclined to look at tools like fossil again, for it's distributed issue tracking and wiki capability https://fossil-scm.org/home/doc/trunk/www/index.wiki

Fossil is faultless for a team size of one. I've been using it for nearly a decade, doing totally non-optimal things like using versions released years apart on different OSs with the same database. I also ctrl-c it when I spot a typo in a commit message and check in binaries. Never missed a beat.

As headcount goes up I think the inability to locally rewrite history into easily reviewable patches would be sorely missed. So it's git for team stuff and fossil for my own.

Re: GitHub incident 2022-03-23

#193
post #161
post #151

Earlier quoted context omitted.

GitHub Availability Report [1] Service Downtime Core Services Only - Cumulative per Month ( Some months with more than one outage) Jan 2021: 3 hours 53 min Feb 2021: 1 hour 42 min Mar 2021: 4 hours 10 min Apr 2021: 2 hours 20 min May 2021: 10 hours 34 min Jun 2021: 0 min Jul 2021: 0 min Aug 2021: 4 hours 23 min Sep 2021: 0 min Oct 2021: 1 hour 36 min Nov 2021: 2 hours 50 min Dec 2021: 0 min Jan 2022: 26 min Feb 2022:…

So, if my math is right (for 2021 only): 1888 min / 525,600 min = 99.64% uptime. If it was more like 99.80+ I think I would be like "meh", but honestly for the price you pay that's not terrible. Still, for a company at the Microsoft level, it should be 99.80 at least.

This is the same Microsoft that reboots laptops in the middle of teams calls to do hour long update cycles. >99% is implausibly good.

Re: GitHub incident 2022-03-23

#194
post #93

Earlier quoted context omitted.

IIS Server had/has a memory leak in worker threads that many years ago always forced us to restart the server every few days. Starting in 6.0, they added worker thread recycling and made it a mandatory to choose a time period for every thread to be recycled. Why fix the error when you can just restart the service?

Apache prefork had that since forever. Seems just a garbage collect type pattern.

For old-school mod_perl apps setting MaxRequestsPerChild was often a much better ROI than actually finding and fixing the leaks.

Speaking as somebody who's done over a decade of large scale OO applications perl and is actually really good at finding and fixing the leaks, this has often been intellectually aggravating but every time I've set that option instead I rewarded myself with a glass of bourbon for picking the pragmatic choice and then went back to adding (non-leaky) features that were far more useful to the company in question than cleaning up the older code would've been.

Re: GitHub incident 2022-03-23

#195
post #43

I hope they figure out what’s going on every morning. Heard from inside they don’t know why the db dies everyday but restarting it fixes it.

What's "the db"? It sounds like something of small to medium scale if you can just restart it like that. In any case, why not just relocate some vendor engineers on site for a bit? Or, better, why does the vendor not have a small presence in the corner? Sounds like whatever "the db" is it's probably some (objectively) small but very scary thing that's currently on fire and people are trying to figure out how to put i…

[deleted]

Re: GitHub incident 2022-03-23

#196
post #43

I hope they figure out what’s going on every morning. Heard from inside they don’t know why the db dies everyday but restarting it fixes it.

What's "the db"? It sounds like something of small to medium scale if you can just restart it like that. In any case, why not just relocate some vendor engineers on site for a bit? Or, better, why does the vendor not have a small presence in the corner? Sounds like whatever "the db" is it's probably some (objectively) small but very scary thing that's currently on fire and people are trying to figure out how to put i…

Cock crows. DB crashed.

Systemctl restart

mysqld

(Or mariadb, if you pronounce "SQL" as "sequel")

Re: GitHub incident 2022-03-23

#197
post #42

Earlier quoted context omitted.

Does Gitea support some kind of federation / cross-instance PRs? That's the main thing I'd miss from a self-hosted instance, the ease of getting contributions. After all, you don't even need Gitea for pure Git hosting. If you have a server with SSH access, just init a bare repo in a directory, push to that, and you're ready to go. No web UI needed. The reason I'm still using GitHub is not code hosting. It's collabora…

But by using GitHub for "collaboration" you are sacrificing decentralization.

Absolutely, and I genuinely hate that.

But for new open source projects of mine the ease of contribution and user expectation of a github repository are a trade-off worth making even so (I also maintain a self hosted master git repo that I consider the source of truth to -me- but these days it syncs from, rather than to, github, just because of the trade-offs involved).

Re: GitHub incident 2022-03-23

#199
post #197

Earlier quoted context omitted.

But by using GitHub for "collaboration" you are sacrificing decentralization.

Absolutely, and I genuinely hate that. But for new open source projects of mine the ease of contribution and user expectation of a github repository are a trade-off worth making even so (I also maintain a self hosted master git repo that I consider the source of truth to -me- but these days it syncs from, rather than to, github, just because of the trade-offs involved).

And in return you are sacrificing your code and contributors to a Microsoft-based privacy policy. GitLab is better but still not ideal.

Re: GitHub incident 2022-03-23

#200
post #90

Earlier quoted context omitted.

It's almost like people forget that git is a Distributed Version Control System, after all...

GitHub/Lab are for more than just code repo

Can project management features not be made part of a dumb repo on the db side? (Spoiler: yes, and many projects have explored this — setup unfortunately has never been as easy as "we'll invite u to the gh, check ur email".

Perhaps with decentralization push of web3/QR etc, we'll get over the hump.

Post reply on HN