Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced the cost of building Mastodon at Twitter-scale by 100x

blog.redplanetlabs.com

Post reply on HN
Print view View original

81–90 of 376 posts

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#81

Post by xmonkee »

polishdude20 wrote: post #61

"We spent nine person-months building our scalable Mastodon instance. " Nono, you can't say that when later on you say it's built on top of Rama. You literally spent 10 years building the framework to even make this. And yes, you built this in 10k lines of code but how many lines of code is Rama? This seems disingenuous.

Going by their claims, they are showing off their generalizable platform, Rama, by building an application on top of it. The application is an example, not the product. For example, someone implements a Todo app on their hot new javascript framework in 10 mins, your objection would be, "But it took you 2 years to make the framework, so actually it took you 2 years plus 10 mins". Why stop there? It also took many years to build the underlying language, networking layers, infrastructure, processors, materials etc etc. You have to draw the line at the point where the application specific code starts and the generalizable platform ends, no?

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#82

Post by bo1024 »

polishdude20 wrote: post #61

"We spent nine person-months building our scalable Mastodon instance. " Nono, you can't say that when later on you say it's built on top of Rama. You literally spent 10 years building the framework to even make this. And yes, you built this in 10k lines of code but how many lines of code is Rama? This seems disingenuous.

Their point is to show off the power of Rama, I.e. it is possible to build such applications on top with little work.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#83

Post by nathanmarz »

DoesntMatter22 wrote: post #59
phillipcarter wrote: post #34

In the year of our lord 2023, people are still launching immature products with "we built a clone of a tiny subset of Twitter" as their use case? Come on. Twitter is huge because they have to support a huge number of use cases. Using this proprietary framework won't magically make complex use cases go away.

"We recreated a service from 2007 and it's so much faster!"

As we mentioned in the article, Instagram just spent ~25 person-years building Threads which is a barebones clone of Twitter. Not only did we build our instance 30x faster than that, we have way more features like federation, hashtag follows, polls, DMs, global timelines, and more. And Instagram didn't start from scratch as Instagram/Meta already had infrastructure powering similar products.

https://www.washingtonpost.com/technology/2023/07/29/meta-th...

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#84

Post by erlend_sh »

nathanmarz wrote: post #53
softwaredoug wrote: post #42

It’s a massive ask, even if the platform was 100x better, for all developers to give up every programming language and database they’ve ever used to depend on a startups closed source platform for all functionality. It’s hard enough trusting Google or Amazons cloud offerings won’t change. It seems that’s what they’re proposing right? What am I missing?

We're actually not asking anyone to give up anything. First off, it has a simple integration API (which you'll be able to see the details of next week) that allows it to seamlessly integrate with any other backend tool (databases, monitoring systems, queues, etc.). So Rama can be incrementally introduced into any existing architecture. Second, Rama has a pure Java API and is not a bespoke language. So no new language…

So Rama-powered apps need to be written in Java? Or will any JVM language work?

And the Rama core will remain closed-source? That part seems like the toughest sell of all, at a time when the vast majority of developer tooling and backends are open source or at the very least source-available.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#85

Post by hedora »

riffic wrote: post #4

the group involved here may want to be mindful of the Mastodon gGmbH trademarks. Using the Mastodon logo on redplanetlabs.com to pitch a reimplementation of ActivityPub might be seen as infringing. https://joinmastodon.org/trademark removed part about the mastodon subreddit since this is clearly not about the Mastodon software per se.

Any trademark case is going to have to prove that a reasonable person would think this article is from Mastodon gGmbH, or is talking about their product "Mastodon".

The top of the page reads "Red Planet Labs", the title of the article is "How we reduced the cost of building Twitter at Twitter-scale by 100x" and the first line of the article is "We built a Twitter-scale Mastodon instance from scratch in only 10k lines of code."

No reasonable person is going to think that this article has anything to do with the official Mastodon software, so there's no trademark issue here.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#86

Post by buro9 »

Measuring "Twitter Scale" by tweets per second seems to be not how I would measure it.

Updates per second to end users who follow the 7K tweets per second seems more realistic, it's the timelines and notifications that hurt, not the top of ingest tweets per second prior to the fan out... and then of course it's whether you can do that continuously so as not to back up on it.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#87

Post by [deleted] »

polishdude20 wrote: post #61

"We spent nine person-months building our scalable Mastodon instance. " Nono, you can't say that when later on you say it's built on top of Rama. You literally spent 10 years building the framework to even make this. And yes, you built this in 10k lines of code but how many lines of code is Rama? This seems disingenuous.

[deleted]

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#88

Post by 2Gkashmiri »

whats the server specs of this demo running at?

is it baremetal?

vps?

how about doing a comparison on consumer grade vps like 1 vcpu/4GB ram setup comparison between your product and mastodon or pleroma for example?

i mean sure you can build a twitter scale product but federation means people can do that on their own and with your tech, they dont have to worry about scaling issues.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#89

Post by roguas »

nathanmarz wrote: post #49
LeifCarrotson wrote: post #12

> How is it possible that we’ve reduced the cost of building scalable applications by multiple orders of magnitude? > You can begin to understand this by starting with a simple observation: you can describe Mastodon (or Twitter, Reddit, Slack, Gmail, Uber, etc.) in total detail in a matter of hours. It has profiles, follows, timelines, statuses, replies, boosts, hashtags, search, follow suggestions, and so on. It doe…

Building Twitter/Mastodon *not at scale* isn't that hard and certainly doesn't take 200 person-years. Building it *at scale* is a completely different story. Remember the fail-whale? That was years of Twitter struggling to scale their product. That said, as we described in the post our implementation of Mastodon is less code than Mastodon's official implementation. So not only is Rama orders of magnitude more efficie…

Well since you use clojure, you probably know that to have small codebase, people often pick clojure. Going from point A to point Z quickly is rarely a goal for startups, going through A.. B... C... quickly, is the goal. I am still looking through all this, but a thought of having to bet on some java api + hope and pray it will jump over all unknown hoops, hm.

Comparisons to twitter are unfair, twitter is not really technical gem or is it? It's pretty impressive to build it with 3 ppl in 3 months, but hmm also seems feasible using other tech, given all blueprints are out there.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#90

Post by [deleted] »

phillipcarter wrote: post #34

In the year of our lord 2023, people are still launching immature products with "we built a clone of a tiny subset of Twitter" as their use case? Come on. Twitter is huge because they have to support a huge number of use cases. Using this proprietary framework won't magically make complex use cases go away.

[deleted]
Post reply on HN