...because it's only been a week and it takes longer for things to rot badly?
Why Twitter didn’t go down: From a real Twitter SRE
151–160 of 1001 posts
Re: Why Twitter didn’t go down: From a real Twitter SRE
#152I think the real question is: Twitter grew 3x on the headcount front with a flat stock price over the course of less than 5 years. What exactly where these thousands of employees actually doing and why did the previous CEO think what they were doing was worth hiring them for? That's just basic accountability from a stock holder or employee perspective. That's apparently a ton of money being wasted on nothing at all.
> Twitter grew 3x on the headcount front There were multiple executives making $10m/yr+ There were board members There were shareholders Why did all of them not stop this headcount increase if it's as easily reduced as "too much headcount bad, smaller headcount good"? These are paid professionals who are supposedly wealthy, good at their jobs, smart, informed, etc. How can us commenters on HackerNews sit from our arm…
> How can us commenters on HackerNews sit from our armchair and say "ah, goofballs should've just not let headcount get so high!"
The cliche HN comment on sites like Twitter (and many, many others, any time headcount comes up) has always been "why do they need so many people?" I've mostly dismissed it the same way I dismiss "I could build Uber in a weekend," but with every other tech giant laying people off, maybe I shouldn't. Maybe the effect of all that extra money sloshing around in the system was to incentivize hiring everyone to make sure you didn't accidentally get a false negative, and not all of those hires were good ones.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#153Great post. Thanks for the insight on how mature Twitter’s Cache infrastructure tooling is. Having done similar work at a different place where you’re exposed to the realities of racks and racks of machines I could appreciate the evolution of the stack and knocking off problems one by one.
I suspected that the hopes and fears of Twitter’s quick demise were overstated and shared[0] some of those thoughts last week, on Twitter of course.
Also hi from another infrastructure person in SD!
0 - https://twitter.com/bitsandhops/status/1593637241527578625
Re: Why Twitter didn’t go down: From a real Twitter SRE
#154I think people's expectations became so exaggerated it was inevitable they wouldn't be lived up to. I'm sure Twitter will experience degradation from the drastic cost-cutting, but it was never going to happen overnight and I'm not sure why news outlets were saying that (except that their sources were employees with slightly inflated senses of their own importance, which we're all sometimes guilty of). And people beca…
To me the analogy here is that the new boss rolled in and sold all the fire extinguishers. That by itself doesn't set the building on fire - it doesn't even increase the chances of a fire occurring on any given day. But when one does...
Definitely stealing this as the right way to frame the issue. What would normally be a small kitchen mistake turns into no longer having an apartment complex.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#155I think the real question is: Twitter grew 3x on the headcount front with a flat stock price over the course of less than 5 years. What exactly where these thousands of employees actually doing and why did the previous CEO think what they were doing was worth hiring them for? That's just basic accountability from a stock holder or employee perspective. That's apparently a ton of money being wasted on nothing at all.
> Twitter grew 3x on the headcount front There were multiple executives making $10m/yr+ There were board members There were shareholders Why did all of them not stop this headcount increase if it's as easily reduced as "too much headcount bad, smaller headcount good"? These are paid professionals who are supposedly wealthy, good at their jobs, smart, informed, etc. How can us commenters on HackerNews sit from our arm…
It's not goofballs. It's generally misaligned incentives. Managing a 10,000 org leads to better job prospects than a 1000 person org, than a 100 person org, than a pizza box team.
Organizations tend to bloat.
Random, rapid cuts might not be the fix here, but headcount was too high.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#156I think people's expectations became so exaggerated it was inevitable they wouldn't be lived up to. I'm sure Twitter will experience degradation from the drastic cost-cutting, but it was never going to happen overnight and I'm not sure why news outlets were saying that (except that their sources were employees with slightly inflated senses of their own importance, which we're all sometimes guilty of). And people beca…
To me the analogy here is that the new boss rolled in and sold all the fire extinguishers. That by itself doesn't set the building on fire - it doesn't even increase the chances of a fire occurring on any given day. But when one does...
He also sold all the smoke detectors.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#157...because it's only been a week and it takes longer for things to rot badly?
I think this post is aimed at non-technical "RIP Twitter" posters from last week who are surprised it didn't fall apart in 24 hours.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#158Tangent but I do hope that Musks trim down results in orgs that have less “executives” and a layered cake of a org structure, and more autonomous small teams that execute on shared overarching initiatives.
I really don’t understand why so many tech companies have like 8 layers of engineering levels. If the argument is that you need more money so more levels, just have a bigger band. Don’t chase titles they don’t mean shit. Not to mention the management stacks that seem to just hang out in meetings and take pvt. I haven’t worked at a proper startup but I’ve been on projects where a dozen or so people rebuilt apps used by tens of millions of people in a few months, or launched completely new applications for bigger companies.
Now that I work in a tech company (not big tech, but still a multibillion dollar corp) I’ve noticed that since IPO we have added a ton of bureaucracy whereas back in the day we were small teams building completely new and at times complex features. Literally was in a meeting earlier this month where people were patting each other on the back because we added a single attribute to a table. I’m obviously reducing the entire initiative to a small thing but it kind of explains all we had to do. It’s soul crushing but with the economy the way it is I must deal with it. Hell I’m down to show up next Monday and work for Elon if he wants a go getter. At least I’ll get to DO stuff. Or any other startup in SF if they are hiring.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#159I know most of my comment is going to be redundant, but I need to say it in order to point to it an year or two from now as an "I told you so!" Twitter is going to be here for a long long time. It's not going to suddenly shut down, it's not going to slowly decline, there is not going to be mass abandonment. Elon might be cocky, but at the end of the day he is a successful businessman. He didn't just sent out that loy…
What kind of woke ideology exactly? Sorry not familiar the current climate in USA.
Re: Why Twitter didn’t go down: From a real Twitter SRE
#160I can't help but feel it might not be legal to post all those details.
given that it’s showing Twitter in good light, it would be super silly to pursue anything IMO, even if it were illegal.
What would be a realistic fine in terms of damages (in $$)?
You could argue its actually negative damages.