Live data from Hacker News

We reduced the cost of building Mastodon at Twitter-scale by 100x

blog.redplanetlabs.com

101–110 of 376 posts

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#101

ctrl+f "ads" ctrl+f "monetization" ctrl+f "moderation" ctrl+f "existing infrastructure" ctrl+f "personalization" etc etc Yeah about what I expect from a "we rebuilt twitter for cheap" post. There's no point to the comparisons with the Twitter codebase size/cost. It completely distracts from what is probably a perfectly fine project.

The point is that much of that should be unnecessary to sustain the service because hosting costs are significantly (presumably 100x) cheaper.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#102

Earlier quoted context omitted.

We're actually not asking anyone to give up anything. First off, it has a simple integration API (which you'll be able to see the details of next week) that allows it to seamlessly integrate with any other backend tool (databases, monitoring systems, queues, etc.). So Rama can be incrementally introduced into any existing architecture. Second, Rama has a pure Java API and is not a bespoke language. So no new language…

So Rama-powered apps need to be written in Java? Or will any JVM language work? And the Rama core will remain closed-source? That part seems like the toughest sell of all, at a time when the vast majority of developer tooling and backends are open source or at the very least source-available.

Any JVM language should work. We've built modules with Clojure.

We're keeping it closed-source for now.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#103
post #76

Earlier quoted context omitted.

> Second, Rama has a pure Java API and is not a bespoke language. So no new language needs to be learned. Isn't Mastodon a Ruby On Rails application?

The article says they re-wrote Mastodon from scratch (probably the backend piece). I'm guessing in Java.

Yes, it's 100% written in Java.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#105

Noticeably missing are any details about concurrency control and replication or recovery protocols. A Twitter clone is one thing but any sort of application needing ACID Transactions is a whole other beast.

All data on Rama is replicated automatically with a configurable "replication factor". Data written to Rama is not made visible until it's successfully replicated. The documentation we're releasing next week includes a page going into detail in how this works.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#106

"We spent nine person-months building our scalable Mastodon instance. " Nono, you can't say that when later on you say it's built on top of Rama. You literally spent 10 years building the framework to even make this. And yes, you built this in 10k lines of code but how many lines of code is Rama? This seems disingenuous.

Actually took them longer than that even.

They had to invent the computer first, and before that they had to create a universe capable of sustaining both life and computers.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#107

> The instance has 100M bots posting 3,500 times per second at 403 average fanout to demonstrate its scale. Mastodon has to send messages to each instance with a recipient. That server can then fan out to all it's subscribers. The way this point is worded makes me think all the bits are on just a single instance, meaning all the fan out can be dealt with internally without having to do any server-to-server at all. Th…

We implemented federation fully exactly as you described.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#108
I'm excited to see the docs for Rama. But I am also a little scared of the comment " I came to suspect a new programming paradigm was needed" from Nathan.

It's not so much that I think the comment is wrong or anything, but rather that it seems so similar to what I have heard in the past from power-lisp (or Clojure in this case) super-smart engineers.

I feel like we have reached a point in software development where "better" paradigms don't necessarily gain much adoption. But if Rama wins in the marketplace, that will be interesting. And I am quite excited to see what a smart tech leader and good team have been able to grind out given a years-long timeframe in this programming platform space . . .

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#109
Is there a breakdown of effort Twitter spent doing the mastodon-level service (serving a feed of the accounts you are subscribed to) vs everything else like ads, algorithmic feed, moderation, fighting spam, copyright claims, localization, GR, PR, safety, etc?
Post reply on HN