Earlier quoted context omitted.
Your reply is indistinguishable from a religious person exclaiming that "God is omniscient and moves in mysterious ways!" Stop worshipping billionaires.
You could say that if he was new to the party. But he did change the world with his companies already, didn't he?
Why Twitter didn’t go down: From a real Twitter SRE
721–730 of 1001 posts
Re: Why Twitter didn’t go down: From a real Twitter SRE
#722Earlier quoted context omitted.
Twitter had 7,500 employees. most of the roles you mention (security engineers, code reviews, data architecture, HR, internal audit teams, content moderators, scrum master) are not bloat. So the question is what are the other 7000 people doing?
Sitting on beanbags in group therapy sessions talking about imposter syndrome.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#723Earlier quoted context omitted.
John Carmack, "Elon is definitely an engineer. He is deeply involved with technical decisions at spacex and Tesla. He doesn’t write code or do CAD today, but he is perfectly capable of doing so." Kevin Watson, who developed the avionics for Falcon 9 and Dragon and previously managed the Advanced Computer Systems and Technologies Group within the Autonomous Systems Division at NASA's Jet Propulsion laboratory: "Elon i…
Elon also understands deep neural nets a lot more than I think people imagine. He starts with good intuitions and mental models, but also actively asks for technical deep dives, and has very good retention. E.g. I recall teaching him about our use of focal loss in contrast to binary cross-entropy for the object detection neural net (I said it had given us a 5% bump and he asked to know more) and he understood how it…
You know the old saying: "It Is Difficult to Get a Man to Understand Something When His Salary Depends Upon His Not Understanding It."
That's what's happening here.
That engineer that was humiliated publicly for defending a slow-as-molasses JavaScript-heavy microservices Rube Goldberg machine? Hacker News is filled with people just like him that have built near identical software in other orgs.
Understandably they're upset and are looking for any excuse to dismiss Elon's criticism of not just Twitter, but their entire industry.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#724Earlier quoted context omitted.
> As someone paraphrased, a car without breaks and steering wheel works just fine until you hit the first bend. On the other hand, a car without a second and third steering wheel, 20 windscreen wipers, and an oven in the back, keeps running just fine, even after the first bend...
That might be fine but do you want to fly in a plane that only has one engine? What about if it has no fallback if the main sensors freeze?
I've done that lots of times. As do almost all people flying Cessnas, Moonys, Pipers and the like...
Re: Why Twitter didn’t go down: From a real Twitter SRE
#725Earlier quoted context omitted.
How difficult can a platform like Twitter be? I am convinced you can run the entire tech stack with a team of a 100 people.
> I am convinced you can run the entire tech stack with a team of a 100 people Please tell us in detail about the Twitter stack. Because I always find it fascinating how people think they can estimate the effort to maintain it whilst having next to no understanding what so ever of the tech stack.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#726Earlier quoted context omitted.
> removing engineers won't instantly crash the product. It'll happen slowly It's amazing to me how many people following the Twitter saga, some familiar with or actually working in technology, thought that Twitter would crash within days of the engineers being fired. And because it didn't, the job cuts are justified.
> thought that Twitter would crash within days of the engineers being fired Because they work for companies where the product would fail within days of them being fired themselves.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#727Earlier quoted context omitted.
Another thing we noticed at Netflix was that after services didn’t get pushed for a while (weeks), performance started degrading because of things like undiscovered memory leaks, threads leaks, disks filling up. You wouldn’t notice during normal operations because of regular autoscaling and code pushes, but code freezes tended to reveal these issues.
We used to have a horribly written node process that was running in a Mesos cluster (using Marathon). It had a memory leak and would start to fill up memory after about a week of running, depending on what customers were doing and if they were hitting it enough. The solution, rather than investing time in fixing the memory leak, was to add a cron job that would kill/reset the process every three days. This was easier…
Re: Why Twitter didn’t go down: From a real Twitter SRE
#728Re: Why Twitter didn’t go down: From a real Twitter SRE
#729Earlier quoted context omitted.
So should the platform be guided by advertisers? Especially one that’s apparently the de facto public square?
What choice do they have? The company was hovering around profitable (2021 would’ve been without a lawsuit settlement) but that was before they were saddled with a ton of new debt. It’s possible that they could find new revenue models - that pay for checkmark scheme isn’t it but maybe a smart businessman could come up with a better variant - but that takes time and has to be done carefully since they have to keep the…