We reduced the cost of building Mastodon at Twitter-scale by 100x

is there a way to guarantee reading your own writes from a client perspective?

> ...10k lines of code. This is 100x less code than the ~1M lines Twitter

I wish I didn't see this comparison, which is not fair at all. Everyone in their right mind understands that the number of features is much less, that's why you have 10k lines.

Add large-scale distributed live video support at the top of that, and you won't get any close to 10k lines. It's only one of many many examples. I really wish you compare Mastodon to Twitter 0.1 and don't do false advertising

> 100M bots posting 3,500 times per second... to demonstrate its scale

I'm wondering why 100M bots post only 3500 times per second? Is it 3500 per second for each bot? Seems like it's not, since https termination will consume the most of resources in this case. So I'm afraid it's just not enough.

When I worked in Statuspage, we had support of 50-100k requests per second, because this is how it works - you have spikes, and traffic which is not evenly distributed. TBH, if it's only 3500 per second total, then I have to admit it is not enough.

Kinda disappointed by the simulation, where are all the viral posts?

I've been digging around for a while and haven't found any posts with more than 20 faves. The accounts I've found with ~1 million followers have little to no engagement. I want to see how a post with a million faves holds up to the promises of "fast constant time".

I'm especially curious about these queries — fave-count and has-user-faved — since a couple years ago Twitter stopped checking has-user-faved when rendering posts more than a month or so old, so I imagine it was expensive at scale.

You have the option to track the latest update time and, during the minute immediately following this update, direct all reads to come from the leader. Additionally, you could oversee the replication lag among followers and block queries on any follower that lags more than a minute behind the leader.

For the client, it's feasible to retain the timestamp of its most recent write. In this way, the system can ensure that the replica responsible for any reads related to that user incorporates updates at minimum up to that recorded timestamp. If a replica isn't adequately current, the read can either be managed by another replica or the query can wait until the replica catches up. The timestamp might take the form of a logical timestamp, signifying the order of writes (e.g., log sequence number), or it could be based on the actual system clock, where synchronized clocks become vital.

When your replicas are spread across multiple datacenters—whether for user proximity or enhanced availability—there's an added layer of complexity. Requests requiring the leader's involvement must be directed to the datacenter housing the leader.

i always had this question: how realistically is to, having an standard spec and interoperable protocols, for toxic apps of big international tech companies that provides """services""", so instances of implementations can be maintained by municipalities or local tech business and talent with 100x less employees and money? what policies should be in place to achieve that? what would be the challenges? it would be better/healthier? is someone researching such things like transition to sustainable digital services? (sustainable in terms of local labor, privacy, economy, accountability, etc...)

i mean if you think about this as public services not as a business, profit is secondary, and first is just to make the thing better and better for the users, no need for spying , no advertisement, no need for a rich piece of shit somewhere getting a piece of the money paid in your city for every taxi drive, food delivery or to give up privacy to a soulless/faceless entity just because you want to say something publicly or keep in touch with people. there is no disruption from their part, its just an old thing put on the internet, they are just in the middle of everyone's life, just sucking everything they can. is the actual state of affairs "efficient"?

there must be fed up engineers and tech people everywhere with the sad state of IT industry.