Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Twitter didn’t go down: From a real Twitter SRE

matthewtejo.substack.com

Post reply on HN
Print view View original

601–610 of 1001 posts

Re: Why Twitter didn’t go down: From a real Twitter SRE

#601

Post by _3u10 »

andreyk wrote: post #577
Dave3of5 wrote: post #571

I don't think it will fail on a technical level. As this article says lots of engineering has gone in to make the thing pretty resilient. I would also say that there are still enough engineers who work there who can figure out what's gone wrong and "turn it on and off again" or w/e makes it splutter back to life. In terms of changes to the Platform ditto. It's not difficult to make these changes that a team of 100's…

Yep, agree. Twitter's revenue was primarily from ads, and now I'd bet their ads revenue has dropped a huge amount. Given how Musk is behaving now (reinstating Trump etc.), and losing many pivotal sales people, and the firing of a ton of people in charge of dealing with hate speech and such, it seems unlikely advertisers will return now.

I don’t think given their ad sales that the sales people were that pivotal.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#602

Post by drcross »

FranzFerdiNaN wrote: post #480
usgroup wrote: post #467

Earlier quoted context omitted.

I expect it all to work better with Musk in charge. He knows how to make scalable software and he knows about performant teams. That he’s not going to realise these totally obvious first order consequences people are raising seems unlikely.

Everything that is happening with Twitter proves Musk is yet another wealthy idiot who doenst know shit about shit, except how to blow his own horn. Musk is simlpy lucky that Twitter used to have such excellent engineers like this SRE so that the site isnt yet on fire.

Musk lead teams at the only company who can land commercial rockets and the first company to profit from making electric cars. To say he is a wealthy idiot says more about you than it does about Musk.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#604

Post by hnbad »

Moissanite wrote: post #580
Dave3of5 wrote: post #571

I don't think it will fail on a technical level. As this article says lots of engineering has gone in to make the thing pretty resilient. I would also say that there are still enough engineers who work there who can figure out what's gone wrong and "turn it on and off again" or w/e makes it splutter back to life. In terms of changes to the Platform ditto. It's not difficult to make these changes that a team of 100's…

The fact that he was able to "buy" Twitter and yet transfer a significant amount of debt to the company rather than being liable himself is just another sign of how different "rich people accounting" is. He will walk away from this with a bit less theoretical money but no material impact to his life, while thousands of people are having their lives up-ended. How long are we going to keep letting shit like this happen…

Leveraged buyouts are certainly a thing that sounds like it shouldn't be possible the first time you hear of it. It seems extremely odd that you can buy a company with money you don't have and then have the money take on that debt instead of having to take it on yourself.

As I understand it, this works by rounding up potential loans, approaching the board of the company and getting them to sign over ownership of the company for a pittance in return for the shareholders being paid out by the company using the loans you brought in. This feels more like an emergent property of the system (specifically, contract law and how publicly traded companies operate) than how the system is intended to function.

Intuitively this shouldn't be possible as it's acting against the company's own self-interest despite being in the interest of the shareholders (and the buyer), but I think "the company's interest" in practice is defined by "the owners' interest" (and the owners in this case are the shareholders, who sell the company). I guess corporations aren't people after all.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#605

Post by Marazan »

Once thing I've seen that I think is overlooked in the whole "is Twitter's tech stack inefficient" chat is the nature of the service.

A lot of the "Twitter could be run off my laptop" style comments seem to come from people who run, effectively, Read-Only services. They might serve data at thousands of queries per second but the data itself is _slow_. It is video files or music streams or other data that updates infrequently or, if there is dynamic data involved, it comes from a very small number of fixed sources.

Twitter deals with thousands of posts per second that are subject to huge fluctuations in the density _and_ those posts have to be disseminated to the millions of users. Twitter is a two sided problem.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#606

Post by _3u10 »

guiriduro wrote: post #599
Moissanite wrote: post #580

Earlier quoted context omitted.

The fact that he was able to "buy" Twitter and yet transfer a significant amount of debt to the company rather than being liable himself is just another sign of how different "rich people accounting" is. He will walk away from this with a bit less theoretical money but no material impact to his life, while thousands of people are having their lives up-ended. How long are we going to keep letting shit like this happen…

Yeah I love it. I would like to "buy" a multi-million dollar mansion in the Bahamas with big bank money, but have the bank only encumber the property itself, and I get to live in it and pay service staff for my luxury, maybe the odd AirBnB letting to maintain the occasional pretense of repayment, up until the time 'it' can't sustain 'itself' and I walk away debt-free. But apparently I'm not a rich person so that kind…

It’s really easy to get a mortgage for 20% of a property and assign it to a corporation.

In fact most banks will go as high as 80%, especially if the house can rent for the mortgage payment.

And they will definitely do that deal if you’re willing to pay 10% interest.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#607

Post by stefan_ »

larsnystrom wrote: post #578
Dave3of5 wrote: post #571

I don't think it will fail on a technical level. As this article says lots of engineering has gone in to make the thing pretty resilient. I would also say that there are still enough engineers who work there who can figure out what's gone wrong and "turn it on and off again" or w/e makes it splutter back to life. In terms of changes to the Platform ditto. It's not difficult to make these changes that a team of 100's…

Are you saying Twitter is paying ~$1.5B in interest rates every year?

The interest rate of the loans is based on an index + fixed %. If the index goes up, like in an inflationary environment, so do the payments.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#609

Post by adql »

resonious wrote: post #340

> This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I was partly expecting the rest of the article to explain to me why exactly it wasn't just bloat. But it goes on talking about this 1~3-person cache SRE team that built solid infra automation that's really resilient to both hardware and software failures. If anything, the article might actually…

You just need to look at profit and revenue. Revenue grew nicely in last 2-3 years, profit not so much. Bloat is only reason.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#610

Post by yywwbbn »

coldtea wrote: post #535
oxfordmale wrote: post #443

Earlier quoted context omitted.

Let's do a thought experiment and see what functions aren't needed to keep the light on for 30 days: 1) HR 2) Legal 3) Sales 4) Marketing 5) Payroll 6) Admin staff 7) Most of Engineering, other than the bare minimum of L1/L2/3 support. As someone paraphrased, a car without breaks and steering wheel works just fine until you hit the first bend.

> As someone paraphrased, a car without breaks and steering wheel works just fine until you hit the first bend. On the other hand, a car without a second and third steering wheel, 20 windscreen wipers, and an oven in the back, keeps running just fine, even after the first bend...

True, you might be also forced to sell the windshield, doors and the seats to pay the $4 million in interest each day you didn’t have to before
Post reply on HN