Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: GitHub employees what's going on? Why?

news.ycombinator.com

Post reply on HN
Print view View original

171–180 of 280 posts

Re: Ask HN: GitHub employees what's going on? Why?

#171

Post by lokar »

fooey wrote: post #125
sharts wrote: post #74

Earlier quoted context omitted.

I would take what C-level execs say with a grain of salt. Remember these folks hire people with plenty of system design experience. They likely also have the money for hardware resources. If load was an issue these things would have been solved already. Or they are purposely causing millions of dollars of productivity loss daily by not solving this. Something more serious is at play. And I want to hear from the actua…

I think owning GitHub is a prestige play, not necessarily thought of as a profit center. Now however, it's competing for resources directly with the corporations core strategic plans. GitHub having issues is embarrassing, but the shareholders don't really care.

I've always wondered about that. I've suspected the same as you, it's mostly a brand / "ecosystem" thing.

Re: Ask HN: GitHub employees what's going on? Why?

#172

Post by lab14 »

cbeach wrote: post #160
andyjohnson0 wrote: post #134

Github used to be built on mysql / redis / ruby on rails / C / shell, running on dedicated hardware. Microsoft left it like that after they acquired the company. Eventually, though, they decided to migrate the whole thing to Azure. And they were far enough through that to be basically committed... when AI coding started hitting them with much higher workloads. I personally think the reliability problems are more to d…

After their infamous "Fail Whale" outages, Twitter replaced its Ruby on Rails / MySQL stack with modern (at the time) technologies: Scala (functional language running on the JVM) / Finagle (RPC) / Thrift (framework) / Kafka (message queues) and this made Twitter stable under the load of several hundred million users. Ruby is single-threaded, slow with poor GC and memory leaks. As Twitter's Alex Payne said publicly in…

tell that to Shopify that handles 8.15 million requests per second and processes 5.1 million per minute in sales during Black Friday. You scale your architecture, not your programming language.

Re: Ask HN: GitHub employees what's going on? Why?

#173

Post by lab14 »

lexoj wrote: post #39
pluc wrote: post #11

Microsoft. Azure. AI. Pick one!

Also the fact that its written in Ruby on Rails probably doesn’t make it easier, performance wise.

Tell that to Shopify that handles 8.15 million requests per second and processes $5.1M USD per minute in sales during Black Friday. You scale your architecture, not your programming language.

Re: Ask HN: GitHub employees what's going on? Why?

#174

Post by pianopatrick »

Would be funny / interesting / scary if it was another AI related incident like happened with hugging face.

A whole lot of AI agents are presumably using Github as part of a workflow.

Would be story worthy if one of them mis interpreted their instructions and is causing havoc.

Like "Really make sure changes are saved to git" -> agent "I have to hack into github backend to really make sure the changes are saved to disk"

Re: Ask HN: GitHub employees what's going on? Why?

#175

Post by azuanrb »

cbeach wrote: post #160
andyjohnson0 wrote: post #134

Github used to be built on mysql / redis / ruby on rails / C / shell, running on dedicated hardware. Microsoft left it like that after they acquired the company. Eventually, though, they decided to migrate the whole thing to Azure. And they were far enough through that to be basically committed... when AI coding started hitting them with much higher workloads. I personally think the reliability problems are more to d…

After their infamous "Fail Whale" outages, Twitter replaced its Ruby on Rails / MySQL stack with modern (at the time) technologies: Scala (functional language running on the JVM) / Finagle (RPC) / Thrift (framework) / Kafka (message queues) and this made Twitter stable under the load of several hundred million users. Ruby is single-threaded, slow with poor GC and memory leaks. As Twitter's Alex Payne said publicly in…

It’s 2026 and we’re still benchmarking Ruby using Twitter’s 2008 architecture. It was less about a language choice and more about years of architectural redesign. I thought we were past this by now.

Re: Ask HN: GitHub employees what's going on? Why?

#176

Post by chem83 »

herpdyderp wrote: post #2

non-helpful answer: the "Microsoft Acquires GitHub" line in this graph answers all questions https://damrnelson.github.io/github-historical-uptime/

Seems to be a pattern that infra teams in and around Azure don't know how to deal with age-of-AI levels of demand and struggle when internal politics + technical hubris get in the way: https://news.ycombinator.com/item?id=47616242

Re: Ask HN: GitHub employees what's going on? Why?

#177

Post by samudrijan »

minraws wrote: post #159
aleph_minus_one wrote: post #137

Earlier quoted context omitted.

> The AI boom caught them napping though Microsoft, who invested eyewatering sums into AI, even near to betting the whole company on an AI boom, is surprised about the AI boom?

Microsoft bet everything on possible AGI, no one thought the result would be a bonafide commit printing machine.

PRRRR

Re: Ask HN: GitHub employees what's going on? Why?

#178

Post by chrisjj »

digitalsushi wrote: post #66
evanelias wrote: post #65

Earlier quoted context omitted.

My sense is that a lot of this activity isn't even legitimate use. As one example: my company has a GitHub app, and last night some bot added my app to 1700 repos. Then the bot immediately started rapid-firing commits which each affected hundreds of files, triggering a deluge of GitHub webhooks to my servers. The repo names all matched other GitHub apps, so my sense is this bot had added 1700 GitHub apps to 1700 repo…

there's this attack on youtube where people who dont want their real names known will autoblock comments with their real names; bots will spam comments with every word and note the ones that were not visible. kinda reminds me of it. not my most valuable contribution to forensics, admittedly.

This works on only people dumb enough to block only their own name, right?

Re: Ask HN: GitHub employees what's going on? Why?

#179

Post by dmitrygr »

moralestapia wrote: post #102
traviswingo wrote: post #18

Earlier quoted context omitted.

GitHub is on Azure. Azure is notorious for being terrible. There was a post here recently from an ex-Azure engineer explaining all of this in a multi-part series. Edit: Found the thread: https://news.ycombinator.com/item?id=47616242

Q. how can someone publish an article like that, with their name, etc... Don't they sign NDAs? Do they not care? Are they not enforceable? Very curious to know.

Some people reach a level of DGAF where it will not matter

Some fuck-ups have such long-lasting effects that they outlast the NDAs that would cover them

Re: Ask HN: GitHub employees what's going on? Why?

#180

Post by sosodev »

andyjohnson0 wrote: post #134

Github used to be built on mysql / redis / ruby on rails / C / shell, running on dedicated hardware. Microsoft left it like that after they acquired the company. Eventually, though, they decided to migrate the whole thing to Azure. And they were far enough through that to be basically committed... when AI coding started hitting them with much higher workloads. I personally think the reliability problems are more to d…

They didn't leave the architecture alone, right? They shoved React in and created a weird SSR + React frankenstein that is objectively worse in many ways.
Post reply on HN