Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twitter Is DDOSing Itself

sfba.social

Post reply on HN
Print view View original

711–720 of 1001 posts

Re: Twitter Is DDOSing Itself

#711

Post by djbusby »

Rimbo wrote: post #676
sigzero wrote: post #622

Earlier quoted context omitted.

> The people at Twitter who understood the system and could predict the side effects were all fired or left. You have zero idea if that is true or not.

You might be right; the person you're replying to might have zero idea. Now me... I know someone personally who was a senior exec for Twitter's software team, who left after Elon's purge. He left because all the people who understood the system and could predict the side effects were fired or left. He'd been with companies going through death spirals before, and had no interest in being involved with another one. So,…

This is still hearsay.

However, even with my limited experience in similar scenarios on smaller scale - it still feels accurate.

Re: Twitter Is DDOSing Itself

#712

Post by glenngillen »

poisonarena wrote: post #704
lolinder wrote: post #627

Earlier quoted context omitted.

About 80% of Twitter was laid off or quit. I think it's a reasonable supposition that a good number of those were critical personnel who felt they could get a better deal somewhere else.

[flagged]

This is such a disingenuous take. Twitter isn’t just a web page that displays the last thing saved to a database. It curates a different version of that for the millions of users reading it. That’s before you think about Spaces, the whole advertising platform that supports it, the abuse systems, and the infrastructure required to do all of that at a scale far beyond some bootcamp clone.

HN discourse should be above this.

Re: Twitter Is DDOSing Itself

#713

Post by gmerc »

berkle4455 wrote: post #654
pschuegr wrote: post #598

Earlier quoted context omitted.

This. I'd bet substantial amounts of money that the evil scraper idea is the result of a) another issue + b) paranoia + c) Musk thinking he understands better than anybody else.

This is a really ignorant take to dismiss scrapers. LLMs operate by having petabytes of conversational training data. Scraping is how OpenAI trained GPT. It’s how all their copycats are trying to do the same. Elon can be a monumental asshat, and he can be self-DDOS’ing, and can be accurate about scraping at the same time. It’s why every single social media platform is heading toward becoming a walled garden.

It’s quite ignorant to assume petabytes of garbage have any value at this point. See Chinchilla

Re: Twitter Is DDOSing Itself

#714

Post by gmerc »

aeyes wrote: post #703
redditor98654 wrote: post #471

Earlier quoted context omitted.

Depends on the scale of the overall system. I have personally seen and attempted to mitigate degenerate cases where these retries overwhelmed the backend so much that the servers were falling behind in simply rejecting the requests. Infact it got so bad because of all those retries at multiple levels from upstream callers that requests were essentially timing out at the TCP buffer/queue before they could be processed…

It is unlikely that a system with the scale of Twitter implements the API rate limiter in the backend. Usually you'd do this as early as possible together with other WAF stuff. If IPs or IP ranges get really annoying we block them on the network level. Big public sites like Twitter obviously need to have this technology. Due to their political content they probably also need sophisticated DDoS protection.

Sometimes people are cheap and don’t pay their cloudflare bill or their engineers.

Re: Twitter Is DDOSing Itself

#715

Post by olalonde »

evan_ wrote: post #333
aeyes wrote: post #272

This bug is very unlikely to be the reason. The rate limiter on the server side is cheap and the frontend bug only gets triggered with the rate limit active. I have seen similar bugs in the systems I oversee because network libraries love to retry requests without sane limitations by default. But I never saw them make our rate limiters sweat. It's slightly more annoying when they hit an API which actually does some e…

I don’t think it’s necessarily saying the self-inflicted DDoS has caused a technical issue that’s forced them to shut down access. I think it’s possible that shutting down anonymous access caused the DDoS, which led to giant spikes in some metric , which led them (Elon) to conclude that there was an uptick in scraping, so they imposed the 600/tweet/day limit to punish scrapers. Seems like either my quota reset or the…

According to Elon, shutting down anonymous access was itself an emergency measure to deal with the DDoS[0]. Twitter did increase the quota significantly from earlier today[1].

[0] https://twitter.com/elonmusk/status/1674865731136020505

[1] https://twitter.com/elonmusk/status/1675260424109928449

Re: Twitter Is DDOSing Itself

#716

Post by photochemsyn »

hvis wrote: post #692
martincmartin wrote: post #373

Earlier quoted context omitted.

The people at Twitter who understood the system and could predict the side effects were all fired or left. My guess is Elon said "the site's too slow!" Engineers noticed that the home feed request was slow. They didn't understand how it worked, had no tools to profile it, and were given an unrealistic deadline to fix it. So about the only thing they could do was issue multiple, parallel requests and hope that at leas…

> My guess is Elon said "the site's too slow!" Seems more like a thinly disguised attempt to force people to purchase subscriptions. Blue checkmarks and all.

That's the first thing that came to mind. It's a pay-to-play move, which is not at all surprising. People seem to forget that fully-featured social media-centric corporations are run for profit, and if they can't get enough advertising to balance the books, then what's left but subscriptions?

Sometimes people buy media corporations because they're interesting in using them to promote their other more lucrative operations (think Jeff Bezos and the Washington Post) so they don't really care about profitability, but I don't think Twitter fits that model, but who knows?

Re: Twitter Is DDOSing Itself

#717

Post by transcriptase »

wpietri wrote: post #600
agentofoblivion wrote: post #561

Earlier quoted context omitted.

[flagged]

Ooh, good idea. People can go build something against the Twitter API! Oh wait! I'm absolutely going to keep enjoying Musk's comeuppance. In this, consider me a traditionalist. As the Greek tragedies show, people have been enjoying the just punishment of hubris for thousands of years.

You can relish in the idea of Twitter having technical issues as if it hurts Elon in some tangible way, but the fact is he could pull the plug on it tomorrow… simply shut Twitter down entirely. Take the entire value of the company as a personal loss, and give everyone working there $1m for their service.

And still be the wealthiest individual on the planet.

He could do it another 5 times and still be a billionaire.

Yes, I’m aware his net worth doesn’t mean liquid assets but the point remains.

Re: Twitter Is DDOSing Itself

#718

Post by mastazi »

poisonarena wrote: post #704
lolinder wrote: post #627

Earlier quoted context omitted.

About 80% of Twitter was laid off or quit. I think it's a reasonable supposition that a good number of those were critical personnel who felt they could get a better deal somewhere else.

[flagged]

back when I was doing personal projects or working at small early-stage startups I used to think the same, but then I've worked at later stage scaleups and even large publicly traded companies and realised that once a project becomes large enough, that prototyping phase that you just described is like 1% or 2% of the total workload. There is so much more to the SDLC that is not just writing code (some examples: liaising with stakeholders, requirements gathering, clients acceptance testing, QA, integration testing, certifications, regulatory frameworks etc. etc. etc. -> all things that take much of your time and involve nearly 0 lines of code written).

Re: Twitter Is DDOSing Itself

#719

Post by lanstin »

Scalene2 wrote: post #484
rs999gti wrote: post #449

Earlier quoted context omitted.

It's still not forced to work, unless going back to your home country is worse than working at Twitter?

Would anyone get a visa to work in another country if it would not be better than being in their home country?

Maybe if they were unsure about which they preferred.

Re: Twitter Is DDOSing Itself

#720

Post by olalonde »

It's an interesting theory but the DDoS predates the decision to disable anonymous access. In fact, that decision was taken in order to mitigate an ongoing DDoS[0][1]. So while it's possible that the questionnable web front end retry logic is not helping things, it's not the root cause.

[0] https://twitter.com/elonmusk/status/1674865731136020505

> Temporary emergency measure. We were getting data pillaged so much that it was degrading service for normal users!

[1] https://twitter.com/elonmusk/status/1674942336583757825

> This will be unlocked shortly. Per my earlier post, drastic & immediate action was necessary due to EXTREME levels of data scraping.

> Almost every company doing AI, from startups to some of the biggest corporations on Earth, was scraping vast amounts of data.

> It is rather galling to have to bring large numbers of servers online on an emergency basis just to facilitate some AI startup’s outrageous valuation.

Post reply on HN