Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Twitter didn’t go down: From a real Twitter SRE

matthewtejo.substack.com

Post reply on HN
Print view View original

641–650 of 1001 posts

Re: Why Twitter didn’t go down: From a real Twitter SRE

#641

Post by pas »

weinzierl wrote: post #464
resonious wrote: post #340

> This left a lot wondering what exactly was going on with all those engineers and made it seem like it was all just bloat. I was partly expecting the rest of the article to explain to me why exactly it wasn't just bloat. But it goes on talking about this 1~3-person cache SRE team that built solid infra automation that's really resilient to both hardware and software failures. If anything, the article might actually…

No matter if it was or not and for better or worse: If Twitter survives this without any major harm it will have profound consequences for the whole software industry.

That's tiny bit of an exaggeration, maaaybe. Maybe completely prophetic, though!

That said, Instagram was run by just a dozen people back then, while it had hundreds of millions of users, right? So it's not a new data point.

Storing, retrieving, indexing, managing 280 char blobs (with links, threads, embeds) is not exactly the most hardcore of a problem domains.

Microblogs are the typical tutorial topic, and twitter's only "innovation" was to increase the text limit.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#642

Post by hef19898 »

Moissanite wrote: post #580
Dave3of5 wrote: post #571

I don't think it will fail on a technical level. As this article says lots of engineering has gone in to make the thing pretty resilient. I would also say that there are still enough engineers who work there who can figure out what's gone wrong and "turn it on and off again" or w/e makes it splutter back to life. In terms of changes to the Platform ditto. It's not difficult to make these changes that a team of 100's…

The fact that he was able to "buy" Twitter and yet transfer a significant amount of debt to the company rather than being liable himself is just another sign of how different "rich people accounting" is. He will walk away from this with a bit less theoretical money but no material impact to his life, while thousands of people are having their lives up-ended. How long are we going to keep letting shit like this happen…

You were not around during the period of highly leveraged take overs starting in the 80s, were you? That is classical hedge fund behavior, if done right. The fact that Musk lending banks only get around 60 cents on the dollar on the market for the loans they gave Musk shows that it might not have been done "right" when it comes to Twitter.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#643

Post by peoplefromibiza »

kakwa_ wrote: post #524
holoduke wrote: post #506

Earlier quoted context omitted.

How difficult can a platform like Twitter be? I am convinced you can run the entire tech stack with a team of a 100 people.

That's likely optimistic. Maybe the user facing site, but that's just the tip of the iceberg. There are plenty of internal/backend/restricted systems to support and/or monetize this part. And that's not counting the huge number of support people & moderators needed.

WhatsApp, 50 engineers, > 1 billion users daily, sold to Facebook for $19 billions.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#644

Post by watt »

_3u10 wrote: post #593
Dave3of5 wrote: post #571

I don't think it will fail on a technical level. As this article says lots of engineering has gone in to make the thing pretty resilient. I would also say that there are still enough engineers who work there who can figure out what's gone wrong and "turn it on and off again" or w/e makes it splutter back to life. In terms of changes to the Platform ditto. It's not difficult to make these changes that a team of 100's…

If you consider that engineers cost about $1000 a day and he got rid of 4,000 of them… oddly it works out to about $4 million per day. I’m not an expert in math but it’s seems pretty possible. Plus they’ll have a lot more impressions to sell now that people are allowed to speak. Ad rates might drop but then someone will write an article about how they are getting great CPC on Twitter and everything will be back to no…

Dead cat bounce. Engineers are paid $1000 a day because they create value to the tune of $10 000 a day. So maybe you saved $4 million a day in very short term, but now the momentum is gone, and soon you will not be earning $40 million a day from the value that now was not being created, just in a "few" short months.

I wish Elon the best, but he could have hired his own team of "hardcore" engineers and put them to work 80 hours a week, spending a fraction of the price that was paid.

What a waste.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#645

Post by ddalex »

hnbad wrote: post #604
Moissanite wrote: post #580

Earlier quoted context omitted.

The fact that he was able to "buy" Twitter and yet transfer a significant amount of debt to the company rather than being liable himself is just another sign of how different "rich people accounting" is. He will walk away from this with a bit less theoretical money but no material impact to his life, while thousands of people are having their lives up-ended. How long are we going to keep letting shit like this happen…

Leveraged buyouts are certainly a thing that sounds like it shouldn't be possible the first time you hear of it. It seems extremely odd that you can buy a company with money you don't have and then have the money take on that debt instead of having to take it on yourself. As I understand it, this works by rounding up potential loans, approaching the board of the company and getting them to sign over ownership of the…

The question is who is left holding the bag? If the stakeholders get paid, they are happy. The new owner is debt free, and have an ownership, so that's good. It seems to me that it may go two ways: either the lenders will be left holding the empty bag, if they can't make the money back - and it's their fault for doing a bad due diligence; OR the company gets sold off piece by piece to satisfy creditors (if they can't make the business work), which again a good thing - what purpose serves a company that cannot produce a profit ?

Re: Why Twitter didn’t go down: From a real Twitter SRE

#646

Post by hef19898 »

drcross wrote: post #592
Moissanite wrote: post #580

Earlier quoted context omitted.

The fact that he was able to "buy" Twitter and yet transfer a significant amount of debt to the company rather than being liable himself is just another sign of how different "rich people accounting" is. He will walk away from this with a bit less theoretical money but no material impact to his life, while thousands of people are having their lives up-ended. How long are we going to keep letting shit like this happen…

Debts from where? Any evidence of this at all? It seems like fantasy considering how much money the rest of Musks companies have in the bank.

There is Elon's private wealth, mostly in tesla stock from what I understand. The cash and assets from the companies at which he is (majority) owner, CEO or a combination thereof are not his, they belong to his companies, which are separate legal entities.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#647

Post by rzz3 »

motoxpro wrote: post #598
holoduke wrote: post #506

Earlier quoted context omitted.

How difficult can a platform like Twitter be? I am convinced you can run the entire tech stack with a team of a 100 people.

If twitter, a top 10 viewed site in the world, isn't difficult, what is?

To be honest, growth. I work at a tech company of a similar size, and keeping the lights on, so to speak, from a systems perspective is something we can do with a few engineers. By far the bulk of employees at my company are in R&D, sales, business operations, infrastructure growth, etc. If you build a resilient architecture, keeping something like Twitter simply running wouldn’t take that many people.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#648

Post by no_op »

thefz wrote: post #515
memish wrote: post #326

Earlier quoted context omitted.

John Carmack, "Elon is definitely an engineer. He is deeply involved with technical decisions at spacex and Tesla. He doesn’t write code or do CAD today, but he is perfectly capable of doing so." Kevin Watson, who developed the avionics for Falcon 9 and Dragon and previously managed the Advanced Computer Systems and Technologies Group within the Autonomous Systems Division at NASA's Jet Propulsion laboratory: "Elon i…

> "Anyone who actually writes software, please report to the 10th floor at 2 pm today. Before doing so, please email a bullet point summary of what your code commands have achieved in the past ~6 months, along with up to 10 screenshots of the most salient lines of code" Actual quote. Anyone using the term "code commands" comes out a little detached from programming reality, let alone the rest of this request, it is o…

"Code commands" is very plausibly an autocomplete flub of what was supposed to be "code commits." When I type "code comm" my iPhone offers up "commands" as the completion.

I've seen a lot of mockery of this request, but I suspect people aren't considering the wide variance in employee quality that can exist within a mismanaged organization. What Musk was asking for here wouldn't be a good way to evaluate skilled, conscientious developers, but it would be a pretty effective way to rapidly identify people who are basically incompetent or just aren't really doing anything.

Re: Why Twitter didn’t go down: From a real Twitter SRE

#650

Post by fatneckbeardz »

throwawaylinux wrote: post #109
hairofadog wrote: post #105

Regardless of what’s up with the tech stack, I find it difficult to want to be a part of whatever this is: https://mobile.twitter.com/elonmusk/status/15945006557246095... (screenshot: https://raw.githubusercontent.com/aboxwithrocksinit/test-buc... ) If this is the new town square, you can forward my mail to a cabin in the woods.

I think that's pretty hilarious. It has caught the ire of a lot of furious unhinged puritans too, which has actually been the funniest thing about it for me.

No post body was provided.
Post reply on HN