Live data from Hacker News

Twitter Is DDOSing Itself

sfba.social

601–610 of 1001 posts

Re: Twitter Is DDOSing Itself

#601
post #282

Perfect example of why you don’t just fire all your senior engineers. You can’t debug and root cause if you do. So you end up adding hacky point fixes

I definitely agree. I do think the major issue at Twitter is more managerial than engineering though. They likely could have run Twitter at the same output with the current team. The problem is a CEO who is learning on the job by altering basically everything at a superfast pace. The root cause is business & feature experimentation at scale with a tight runway & no executive oversight.

To say he's learning on the job is speculation on your part. He's on the job, I'll give you that.

Re: Twitter Is DDOSing Itself

#602

Earlier quoted context omitted.

That's a non-sequitir. You can be at the top of your field and not completely understand a complex system. Also, the right people may not have even been involved in the implementation of this "feature".

> You can be at the top of your field and not completely understand a complex system. The tweet reads: "Twitter is firing off about 10 requests a second to itself to try and fetch content that never arrives because Elon's latest genius innovation is to block people from being able to read Twitter without logging in." Does that strike you as complex? I mean, surely they had the context (need to log in) because it was…

I don't know what kind of systems you worked on in your career, but even simpler systems with smaller userbases than twitter are quite complex if you are new to it.

Twitter serves their service to the entire world, with multiple layers of systems working in conjunction in order to make things work smoothly. A new engineer that has not been working on it for no more than a couple months would likely be unaware of how the different systems communicate and interact. A change like this will have have a lot of unintended consequences, and not having a senior engineer with lots of context leading the change will undoubtedly cause these kinds of issues.

Re: Twitter Is DDOSing Itself

#603

Earlier quoted context omitted.

If executives are throwing you under a bus, make an exit. There’s no coming back from a clown show like that.

I have a general rule that I make proactive, diligent efforts to inform and correct, and I do it 3 times to 3 different groups in leadership, and if that doesn't work, I'm out. I'm about 2 for 8 but you gotta try sometimes.

You don't happen to have a blog or something where you outline, even if it's without tons of detail or being too specific, what these incidents looked like, do you? These feel like (hopefully) rare enough events that I'd love to try to learn from other people's experiences before I run into this kind of situation myself.

Re: Twitter Is DDOSing Itself

#604

Don't interrupt your opponent when they are making a mistake.

Why do people oppose Twitter? It's a social network, you can choose not to use it.

Some people signed up for Twitter, years ago, specifically to avoid other people using their name. I think it was a thread on HN.

Once you are signed up, it's tempting to use it. Then you realize it's a bad idea to use it, but it takes self-control.

Now, with the current leadership, who knows if they will purge inactive accounts and/or reissue IDs.

I don't know if this scenario accounts for much of the antagonism, but it is one possibility. It is possible that some people don't like it, but can't let go of it completely, and the new management creates uncertainty and doubt that never ends.

Re: Twitter Is DDOSing Itself

#605
post #339

Earlier quoted context omitted.

What do you mean?

Jitter is a little randomness in how long clients wait between retries. It ensures that you don't have a "thundering herd" all retrying at the same time. Imagine if your API used exponential backoff of [1s, 2s, 4s, 8s, ...] and a large group of requests gets a retryable error at t=0. They will all retry at exactly t=1, t=2, etc. If the group is large enough that repeated surge of requests can knock you offline. There…

Also useful in caching mechanisms.

Re: Twitter Is DDOSing Itself

#606

Earlier quoted context omitted.

The people at Twitter who understood the system and could predict the side effects were all fired or left. My guess is Elon said "the site's too slow!" Engineers noticed that the home feed request was slow. They didn't understand how it worked, had no tools to profile it, and were given an unrealistic deadline to fix it. So about the only thing they could do was issue multiple, parallel requests and hope that at leas…

> The people at Twitter who understood the system and could predict the side effects were all fired or left. Are you saying the engineers who are now at Twitter don’t have the right skills?

Are the very least they have terrible judgment considering they still work at Twitter (excluding people dealing with visa issues)

Re: Twitter Is DDOSing Itself

#607
post #419

Earlier quoted context omitted.

Could someone report the error at press@twitter.com and see what they think about it?

Quite the knee slapper. Thanks. I hope you didn’t spend many brain cycles on it.

Why do you take people criticizing Elon Musk so personally?

Re: Twitter Is DDOSing Itself

#608

Earlier quoted context omitted.

> The story that anyone who knew anything was let go is just that, a story, as the number of employees by year shows Twitter, and other tech companies in the same period, overhired and then went back to pre-2020 levels. Citation needed. Show me the 2020 Twitter headcount. Show me the 2023 Twitter headcount. > The only difference between then and now is that there is a big personality at the top who now personifies ev…

[flagged]

Why should a victim of hate speech spread the hate being spewed at them?

Re: Twitter Is DDOSing Itself

#609

> If you missed the initial news of the impending showdown, Platformer reported on June 10th that Twitter had been refusing to pay Google for its cloud services ahead of their contract’s June 30th renewal date. > Twitter’s Google Cloud contract dates back to 2018. https://www.engadget.com/twitter-has-supposedly-started-payi... Oh... this explains all this fiasco :facepalm:

https://www.reuters.com/technology/twitter-resumes-paying-go... That's from a week ago.

The sourcing on that story is a single unnamed person whereas both companies refused to comment. Seems like there could be more to the story than "Twitter started paying again and everything's fine."

Re: Twitter Is DDOSing Itself

#610
post #340
post #126

Speaking from very painful, personal experience, few things are more agitating than being forced to execute on something you fully know is a horrible idea, especially when you tried and failed to communicate this fact to the individual pushing you to go against your best judgement. Even more so when that person later loudly proclaims that they never made such a request, even when provided with written proof. I can of…

[flagged]

It's really not, and for quite a lot of people.

This might be kind of true for a software engineer, but not for most people. Especially since if you quit or get fired, and can't find another job, you have no health insurance.

Post reply on HN