I wonder why timelines aren't implemented as a hybrid gather-scatter choosing strategy depending on account popularity (a combination of fan-out to followers and a lazy fetch of popular followed accounts when follower's timeline is served). When you have a celebrity account, instead of fanning out every message to millions of followers' timelines, it would be cheaper to do nothing when the celebrity posts, and later…
Why do they "insert" even non-celebrity posts into each follower's timeline? That is not intuitive to me.
Miscommunication leads to bad outcomes. One missed message out of order could easily lead to a fight, a lawsuit, a flash mob, threats of violence - that then need to be taken seriously, swatting, DOXxing, etc... Msg 1: I hate ___insert_controversal_person_category_here___ Msg 2: Is the kind of statement that really sets me off Msg 1 has a very different meaning if you don't see Msg 2.
This can already happen without help from the platform.
Sure, but that doesn't mean the platform should make it worse.
Trying to have a conversation on flaky platform is hell.
AWS has a cool general approach to this problem (one badly behaving user effecting others on their shard) https://aws.amazon.com/builders-library/workload-isolation-u... The basic idea is to assign each user to multiple shards, decreasing the changes of another user sharing all their shards with the badly behaving user. Fixing this issue as described in the article makes sense, but if they did shuffle sharding in the…
I think shuffle sharding is beneficial for read-only replica cases, not for writing scenarios like this. You'll have to write to the primary and not to a "virtual node". Right? Or am I understand it incorrectly? I just read that article now.
That blows my mind, I would expect maybe 1 or 2 passengers per plane at most. I'm trying to think of what factors would cause that many no-shows, it has to be mostly missed connections? I can't imagine spending hundreds of dollars and just not showing up.
A friend of mine works for a Management Consultancy firm and they have full flex tickets if they miss the 8pm flight home they can take the next one or fly back the next morning. All without additional fees. So I believe business travel is the biggest factor when it comes to missed flights. Side note: His employer is the biggest client of a major European airline.
No additional fees, but the cost of the ticket is typically sky-high. In many cases the company could purchase 3 restricted-fare tickets for less than the unrestricted fare. (And also, the consultants want to get home, they HATE staying extra nights, so they have inherent motivation to catch the original flight). You'd think that management consulting firms, you know, who are supposed to be good at optimization and reducing costs, would have figured this out.
I wonder why timelines aren't implemented as a hybrid gather-scatter choosing strategy depending on account popularity (a combination of fan-out to followers and a lazy fetch of popular followed accounts when follower's timeline is served). When you have a celebrity account, instead of fanning out every message to millions of followers' timelines, it would be cheaper to do nothing when the celebrity posts, and later…
This problem is discussed in the beginning of the Designing Data-Intensive Applications book. It's worth a read!
I wonder why timelines aren't implemented as a hybrid gather-scatter choosing strategy depending on account popularity (a combination of fan-out to followers and a lazy fetch of popular followed accounts when follower's timeline is served). When you have a celebrity account, instead of fanning out every message to millions of followers' timelines, it would be cheaper to do nothing when the celebrity posts, and later…
Why do they "insert" even non-celebrity posts into each follower's timeline? That is not intuitive to me.
To serve a user timeline in single-digit milliseconds, it is not practical for a data store to load each item in a different place. Even with an index, the index itself can be contiguous in disk, but the payload is scattered all over the place if you keep it in a single large table.
Instead, you can drastically speed up performance if you are able to store data for each timeline somewhat contiguously on disk.
I wonder why timelines aren't implemented as a hybrid gather-scatter choosing strategy depending on account popularity (a combination of fan-out to followers and a lazy fetch of popular followed accounts when follower's timeline is served). When you have a celebrity account, instead of fanning out every message to millions of followers' timelines, it would be cheaper to do nothing when the celebrity posts, and later…
This is probably what we'll end up with in the long-run. Things have been fast enough without it (aside from this issue) but there's a lot of low-hanging fruit for Timelines architecture updates. We're spread pretty thin from a engineering-hours standpoint atm so there's a lot of intense prioritization going on.
I guess I hadn’t considered that search engines could be reranking pages on the fly as I click them. I’ve been seeing my DuckDuckGo results shuffle around for a while now thinking it’s an awful bug. Like I click one page, don’t find what I want, and go back thinking “no, I want that other result that was below” and it’s an entirely different page with shuffled results, missing the one that I think might have been goo…
I don't use DDG, but in my (very limited, just now) testing it doesn't seem to shuffle results unless you reload the page in some way. Is it possible you're browser is reloading the page when you go back? If so, setting DDG to open links in new tabs might fix this problem.
Interesting. Maybe something in my configuration is affecting it. I’ll have to look into it
A friend of mine works for a Management Consultancy firm and they have full flex tickets if they miss the 8pm flight home they can take the next one or fly back the next morning. All without additional fees. So I believe business travel is the biggest factor when it comes to missed flights. Side note: His employer is the biggest client of a major European airline.
No additional fees, but the cost of the ticket is typically sky-high. In many cases the company could purchase 3 restricted-fare tickets for less than the unrestricted fare. (And also, the consultants want to get home, they HATE staying extra nights, so they have inherent motivation to catch the original flight). You'd think that management consulting firms, you know, who are supposed to be good at optimization and r…
... or perhaps they prioritize ensuring their consultants feel like they're being looked after - replacing them when they find another job is very expensive.
An airline reservation system has to be perfect (no slack in today's skies), a hotel reservation can be 98% perfect so long as there is some slack and you don't mind putting somebody up in a better room than they paid for from time to time. A social media system doesn't need to be perfect at all. It was clear to me from the beginning that Bluesky's feeds aren't very fast, not like they are crazy slow, but if it saves…
> airline reservation system has to be perfect (no slack in today's skies)
The slack just gets moved. Airlines oversell by about 8 percent. All
systems need some slack in them. Isn't that kinda Bob's Law or
something?