Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced the cost of building Mastodon at Twitter-scale by 100x

blog.redplanetlabs.com

Post reply on HN
Print view View original

331–340 of 376 posts

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#331

Post by philipwhiuk »

dpweb wrote: post #259
nathanmarz wrote: post #168

Earlier quoted context omitted.

We're comparing just to the original consumer product, which is about the same as Mastodon is today. That's why we said "original consumer product" and not "Twitter's current consumer product". Mastodon actually has more features than the original Twitter consumer product like hashtag follows, global timelines, and more sophisticated filtering/muting capabilities. Some people argue it's not so expensive to build a sc…

Built some log databases and back end frameworks myself with some of the same concepts. I applaud the creativity in rethinking how back ends should work. Please now do frontends next! :) "But it's not a fully functioning 2023 Twitter!!!" I think some people miss the point. This is not about hey we built a Twitter clone. This is about a POC for a novel app architecture. We need to be constantly examining and re-examin…

> This is not about hey we built a Twitter clone.

Then they shouldn’t have titled it “we built a Twitter clone”

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#332

Post by runeks »

Sounds like Rama is also useful for small scale applications (where high scalability isn’t needed), since it simplifies how they’re implemented.

Is this the case — ie. would a TodoMVC app implemented in Rama also be much simpler than a traditional frontend/backend/database CRUD implementation?

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#333

Post by wink »

Look, I don't want to defend Twitter but ignoring 15 years of changes and the whole journey of scaling and then using the cost op just building a snapshot of the 15y old version is pretty disingenuous.

That's a bit like starting an Oracle clone now and summing up what they spent on developer salaries in the last 40 years. You basically can't not "reduce costs".

And no "the original consumer product" is not a real cop-out, you probably still have tons of people building iterations.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#334

Post by smcl »

RomanPushkin wrote: post #163

> ...10k lines of code. This is 100x less code than the ~1M lines Twitter I wish I didn't see this comparison, which is not fair at all. Everyone in their right mind understands that the number of features is much less, that's why you have 10k lines. Add large-scale distributed live video support at the top of that, and you won't get any close to 10k lines. It's only one of many many examples. I really wish you compa…

Not to mention the phrase "x times less than" doesn't really make sense the way it's often used. For it to make sense you have to reinterpret it to mean something that it doesn't based on being the opposite of "x times more than" (which is also often misused).

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#335

Post by lbalazscs »

KingOfCoders wrote: post #315
RomanPushkin wrote: post #163

> ...10k lines of code. This is 100x less code than the ~1M lines Twitter I wish I didn't see this comparison, which is not fair at all. Everyone in their right mind understands that the number of features is much less, that's why you have 10k lines. Add large-scale distributed live video support at the top of that, and you won't get any close to 10k lines. It's only one of many many examples. I really wish you compa…

The 100x less code always reminds me of a hypothetical Half Life 2 game engine. Then your code would be: StartHalfLife2LikeGame() and you'd replace millions LOC with one line. If there is a perfect match between the framework and your app, there is no code. The more your app diverges from the ideal app the framework was written for, the more code you have.

They claim that "Reddit, Slack, Gmail, Uber, etc" could also be done with a similar level of effort.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#336

Post by jillesvangurp »

alexcpn wrote: post #324

Why choose Java of all languages. Why not something more modern and less verbose like Go or Rust. Just asking as I have worked enough in Java and then spend a lot of time in GC tunining. Granted the code was not that great and from a diverse team with different skill levels causing all the leaks.. But still

They actually used clojure; which is an interesting choice.

GC tuning on the JVM is much less of a topic these days than it used to be. The default garbage collector was changed at some point (G1). It has some configuration options but they come with sane defaults that mostly just work fine and adapt to your memory and cpus. You don't spend a lot (or any) time on tuning this typically. I know I haven't even looked at GC params in many years now. Never had to. And we run on modestly sized vms of 1 or 2 GB typically. This was different 10 or so years ago when G1 was still newish and not default. ZGC was introduced with Java 11 (I think), and aimed at very large heaps. It trades off additional overhead for guaranteeing very low latency. That tradeoff is why it is not default. For most users, G1 without any tuning whatsoever should be fine. Generally, if you are stressing your heap, you get more hardware. And if you are not, the GC should be keeping up just fine.

Anyway, like it or not, the JVM has been a work horse for big data for ages. Things like Hadoop, Kafka, Cassandra, Elasticsearch, etc. all run on it and scale fine (and typically without a lot of GC tuning). The only feasible alternative to the jvm used to be things like C++. Lately, Go and Rust are pretty credible in this space as well and both have had to do a bit of catchup in terms of maturity of tooling, libraries, and language features. Things like generics (Go), async (Rust), etc. are still fairly recent additions and both kind of relevant in a project of this type.

In any case, switching languages is hard for teams and these guys have been around for quite some time. When they started, Rust was a lot less mature than it is now and Go was still pretty new as well. Neither was an obvious choice for this stuff at the time.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#337

Post by KingOfCoders »

lbalazscs wrote: post #335
KingOfCoders wrote: post #315

Earlier quoted context omitted.

The 100x less code always reminds me of a hypothetical Half Life 2 game engine. Then your code would be: StartHalfLife2LikeGame() and you'd replace millions LOC with one line. If there is a perfect match between the framework and your app, there is no code. The more your app diverges from the ideal app the framework was written for, the more code you have.

They claim that "Reddit, Slack, Gmail, Uber, etc" could also be done with a similar level of effort.

Yes, all applications that are aligned with the framework (message spreading from one user to many others) - not that I want to put down their effort in any way.

But this is the core [0] - a simple event location plattform my wife founded hat easily >200k LOC while the core (edit/search/detail page event location) had [0] If building Twitter I perhaps would try their setup, although I have been bitten with "magic" JVM frameworks in the past, e.G. we used one commercially and the license cost went up from 80k to 800k YoY. On top of scaling problems we could do nothing about because the framework was a black box.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#338

Post by jillesvangurp »

runeks wrote: post #330
chc4 wrote: post #47

Earlier quoted context omitted.

This is usually referred to as an "embedded DSL" - you have a DSL embedded in a normal programming language using its first class constructs.

Meh. By this definition all libraries expose an "embedded DSL" — their API. I'm honestly not sure this is a useful definition.

Whether you like it or not; internal DSLs became a thing with Ruby back in the day. And these days things like Kotlin also lend themselves pretty well to creating internal DSLs. Java is not ideal for this. Kotlin and Ruby have a few syntax features that make it very easy.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#340

Post by jwmoz »

dataangel wrote: post #192

I do C++ backend work in a non-web industry and this entire post is Greek to me. Even though this is targeted at developers, you need a better pitch. I get "we did this 100x faster" but the obvious followup question is "how" but then the answer seems to be a ton of flow diagrams with way too many nodes that tell me approximately nothing and some handwaving about something called P-States that are basically defined to…

Agreed, just reading through half of it I have no idea what Rama is.
Post reply on HN