Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced the cost of building Mastodon at Twitter-scale by 100x

blog.redplanetlabs.com

Post reply on HN
Print view View original

281–290 of 376 posts

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#281

Post by ricardobeat »

ethbr1 wrote: post #226
sdwr wrote: post #207

Earlier quoted context omitted.

In a typical architecture, the DB stores data, and the backend calls the DB to make updates and compile views. Here, the "views" are defined formally (the P-states), and incrementally, automatically updated when the underlying data changes. Example problem: Get a list of accounts that follow account 1306 "Classic architecture": - Naive approach. Search through all accounts follow lists for "1306". Super slow, scales…

So... at a high level, early React for data? In other words, letting a framework manage update dependency graph tracking, and then cascading updates through its graph in an optimized manner to enhance performance? Obviously, with tons of implementation difficulties and details, and not actual graph structures, but as a top level analogy.

Not at all, especially because React doesn’t do much dependency tracking on its own and is built for predictable UI updates and not performance.

To be honest any parallel with frontend here is meaningless, reactivity and all the concepts at play have existed long before JS and browsers came along, it’s easier to explain from first principles.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#282

Post by dbish »

nathanmarz wrote: post #185
newaccount74 wrote: post #164

Earlier quoted context omitted.

Considering the length and amount of detail in this blog post, I understand why they would need another week to get the code ready (assuming there will be more docs)

We're releasing 100k words of high-quality documentation next week.

Measuring words and loc is not a great way, imho, to share what you’re doing. In fact, I’d love a much shorter set of documentation now to under and this better. Long docs will probably make it less likely to read.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#283

Post by riffic »

hedora wrote: post #85
riffic wrote: post #4

the group involved here may want to be mindful of the Mastodon gGmbH trademarks. Using the Mastodon logo on redplanetlabs.com to pitch a reimplementation of ActivityPub might be seen as infringing. https://joinmastodon.org/trademark removed part about the mastodon subreddit since this is clearly not about the Mastodon software per se.

Any trademark case is going to have to prove that a reasonable person would think this article is from Mastodon gGmbH, or is talking about their product "Mastodon". The top of the page reads "Red Planet Labs" , the title of the article is "How we reduced the cost of building Twitter at Twitter-scale by 100x" and the first line of the article is "We built a Twitter-scale Mastodon instance from scratch in only 10k line…

there's this though:

https://mastodon.redplanetlabs.com/timeline/local

and this from the gGmbH trademark policy:

> You may not use the Mastodon word mark, or any similar mark, in your domain name, unless you have written permission from Mastodon gGmbH.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#284

Post by softwaredoug »

ThinkBeat wrote: post #208

I am confused. This is meant to be hyped to sell your Rama platform/product/framework? That you have spent 10 years building in secret? During that time you have built a datastore and a Kafke competitor and ? Should not those 10 years be factored into the time it took to develop this technical demo? Is it 100x less code including every LOC in all of Rama? I mean I am sure you picked a use cast that is well suited to…

Kind of reminds me of when FoundationDB came out, they really needed to demonstrate lots of different use cases to prove they were the database storage layer to rule them all... Not just one

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#285

Post by Aeolun »

Aeolun wrote: post #264
nathanmarz wrote: post #252

Earlier quoted context omitted.

We are very open in the post that the core concepts are not new: Individually, none of these concepts are new. I’m sure you’ve seen them all before. You may be tempted to dismiss Rama’s programming model as just a combination of event sourcing and materialized views. But what Rama does is integrate and generalize these concepts to such an extent that you can build entire backends end-to-end without any of the impedan…

> But what Rama does is integrate and generalize these concepts to such an extent that you can build entire backends end-to-end without any of the impedance mismatches or complexity Every time I hear this the reality turns out to be that building anything with this tech is like building something on top of SAP. But I’m also just allergic to any post that says ‘look how amazing’ in general, so I’m a bit prejudiced.

After reading through the post a bit more, I’m inclined to believe it’s not hot air, but I think most of the innovation where is in the management layer, not the ease of application development.

Just looking at the first example tells me that there’s a million ways someone that doesn’t know what they’re doing can mess this up.

If the author of the platform implements some service on their own platform it’s always going to seem simple.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#286

Post by jeremyjh »

hosh wrote: post #206
adventured wrote: post #200

Earlier quoted context omitted.

None of those things get eliminated by decentralization, they get distributed to whatever the point of control / ownership is. Mastodon still requires security, compliance and moderation. And those requirements are going to keep getting more challenging by the year. It'll end up being another reason nobody will want to host content in a decentralized manner, the burden will become obnoxious.

An organization trying to maintain an ISO certification will have drastically different policies and controls than a small shop, or even a hobbyist group. Everyone (theoretically) would be complying to statues in its broadest sense, but jurisdiction, regulations, industry best practices, reporting requirements, and appetite for risk is going to be different from organization to organization. It’s not one-size-fits-al…

The scrutiny from investors, legal and compliance exists because the risks are real, and they don't go away just because there isn't a Serious Business involved. Once someone operating one of these is publicly damaged, the risk will be better understood and marked up accordingly.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#287

Post by elisbce »

The real reason why we can't easily replicate Twitter/Facebook/Google is because we don't have the distributed storage/caching/logging/data processing/serving/job scheduling/... infrastructures that they have built internally that are designed to provide some level of guaranteed SLAs for the desired scale, performance, reliability and flexibility, not because it is hard to replicate the application logic like posting to timelines. That's also why Threads were built by a small team rather quickly -- they already have the battle-tested infras that can scale.

Any attempt to build a simplified version of the ecosystem will face the same fundamental distributed system tradeoffs like consistency/reliability/flexibility/... For example, one of the simplifications may be mixing storage/serving/ETL workloads on the same node. And the consequence is that without certain level of performance isolation, it could impact the serving latency during expensive ETL workload.

For Rama to be adopted successfully, I think it is important to identify areas where it has the most strengths, and low LOCs might not be the only thing that matters. For example, demonstrating why it is much better/easier than setting up Kafka/Spark and a database and build a Twitter clone on top of that while providing similar/better performance/reliability/extensibility/maintainability/... is a much stronger argument.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#288

Post by Aeolun »

ricardobeat wrote: post #281
ethbr1 wrote: post #226

Earlier quoted context omitted.

So... at a high level, early React for data? In other words, letting a framework manage update dependency graph tracking, and then cascading updates through its graph in an optimized manner to enhance performance? Obviously, with tons of implementation difficulties and details, and not actual graph structures, but as a top level analogy.

Not at all, especially because React doesn’t do much dependency tracking on its own and is built for predictable UI updates and not performance. To be honest any parallel with frontend here is meaningless, reactivity and all the concepts at play have existed long before JS and browsers came along, it’s easier to explain from first principles.

I think that’s probably not the case for many new developers that don’t have any exposure to anything not React. Of course ‘react for data’ is entirely misleading, but it may give a decent idea if you don’t have an hour to spend on an explanation.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#289

Post by hosh »

jeremyjh wrote: post #286
hosh wrote: post #206

Earlier quoted context omitted.

An organization trying to maintain an ISO certification will have drastically different policies and controls than a small shop, or even a hobbyist group. Everyone (theoretically) would be complying to statues in its broadest sense, but jurisdiction, regulations, industry best practices, reporting requirements, and appetite for risk is going to be different from organization to organization. It’s not one-size-fits-al…

The scrutiny from investors, legal and compliance exists because the risks are real, and they don't go away just because there isn't a Serious Business involved. Once someone operating one of these is publicly damaged, the risk will be better understood and marked up accordingly.

They sure are, which brings back to the point: how much did this implementation of Mastadon adequately addressed these risks with the reduced code count?

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#290

Post by rasz »

dataangel wrote: post #192

I do C++ backend work in a non-web industry and this entire post is Greek to me. Even though this is targeted at developers, you need a better pitch. I get "we did this 100x faster" but the obvious followup question is "how" but then the answer seems to be a ton of flow diagrams with way too many nodes that tell me approximately nothing and some handwaving about something called P-States that are basically defined to…

Here in video form: Microservices https://www.youtube.com/watch?v=y8OnoxKotPQ
Post reply on HN