Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced the cost of building Mastodon at Twitter-scale by 100x

blog.redplanetlabs.com

Post reply on HN
Print view View original

111–120 of 376 posts

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#111

Post by nathanmarz »

roguas wrote: post #89
nathanmarz wrote: post #49

Earlier quoted context omitted.

Building Twitter/Mastodon *not at scale* isn't that hard and certainly doesn't take 200 person-years. Building it *at scale* is a completely different story. Remember the fail-whale? That was years of Twitter struggling to scale their product. That said, as we described in the post our implementation of Mastodon is less code than Mastodon's official implementation. So not only is Rama orders of magnitude more efficie…

Well since you use clojure, you probably know that to have small codebase, people often pick clojure. Going from point A to point Z quickly is rarely a goal for startups, going through A.. B... C... quickly, is the goal. I am still looking through all this, but a thought of having to bet on some java api + hope and pray it will jump over all unknown hoops, hm. Comparisons to twitter are unfair, twitter is not really…

Well, as mentioned in the post Instagram literally just built and released their own barebones Twitter clone this year, and it took them 25 person years. They were also able to leverage all their existing infrastructure powering similar products.

So I would not say it's remotely feasible to do this in less than one person-year with any other technology.

https://www.washingtonpost.com/technology/2023/07/29/meta-th...

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#113

Post by colatkinson »

jauntywundrkind wrote: post #64
ollien wrote: post #11

Earlier quoted context omitted.

Yeah, I think this is just an ActivityPub server that supports the Mastodon extensions, right? I think we should embrace the fact that the federated world can be diverse, rather than just call everything "Mastodon"

Mastodon has it's own API. It basically offers a very limited ActivityPub API too, but it's own API is very different. And it's a very slim ActivityPub inplementation. For example, I don't think you can do basic things like get an individual post in ActivityPub. This should be easy simple json-ld to get but it's just 404. https://www.w3.org/TR/activitypub/#retrieving-objects

Mastodon for sure supports fetching individual posts over ActivityPub. For example:

    curl -L -H 'accept: application/activity+json' 'https://mastodon.social/users/Gargron/statuses/18614983'
It does have a bunch of stuff that isn't federated though, such as Like counts/collections. And of course it only implements the server-to-server (S2S) part of AP, not the client-to-server (C2S) part.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#114

Post by pmlnr »

jauntywundrkind wrote: post #64
ollien wrote: post #11

Earlier quoted context omitted.

Yeah, I think this is just an ActivityPub server that supports the Mastodon extensions, right? I think we should embrace the fact that the federated world can be diverse, rather than just call everything "Mastodon"

Mastodon has it's own API. It basically offers a very limited ActivityPub API too, but it's own API is very different. And it's a very slim ActivityPub inplementation. For example, I don't think you can do basic things like get an individual post in ActivityPub. This should be easy simple json-ld to get but it's just 404. https://www.w3.org/TR/activitypub/#retrieving-objects

Mastodon. o. not a.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#115

Post by nathanmarz »

clusterhacks wrote: post #108

I'm excited to see the docs for Rama. But I am also a little scared of the comment " I came to suspect a new programming paradigm was needed" from Nathan. It's not so much that I think the comment is wrong or anything, but rather that it seems so similar to what I have heard in the past from power-lisp (or Clojure in this case) super-smart engineers. I feel like we have reached a point in software development where "…

This is why we exposed Rama as a Java API rather than Clojure or our internal language (which is defined with Clojure macros, so it's technically also Clojure). Rama's Java dataflow API is effectively a subset of our internal language, with operations like "partitioners" being implemented using continuations.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#118

Post by afro88 »

Looks amazing and incredibly smart. But I found the LOC and implementation time comparisons to Twitter and Threads very disingenuous. It makes me wonder what other wool will be pulled over our eyes with Rama in future (or important real world details missed / future footguns).

Still super impressive. Reminds me of when I discovered Elixir while building a social-ish music discovery app. Switching the backend from Rails to Elixir felt like putting on clothes that actually fit after wearing old sweats. Rama looks like a similar jump, but another layer up, encompassing system architecture.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#119

Post by cutler »

nathanmarz wrote: post #77
mping wrote: post #72

Congrats on the (kinda) launch. I was curious to see what you guys were up to. The blog post is pretty detailed, and with good insights. Reducing modern app development complexity to mixing data structures sounds like a good abstraction. I'm sure you thought really hard about the building blocks of Rama and you know your problems better than most of the hn crowd. Now, the really hard part becomes selling. If companie…

Rama is written in Clojure :)

"Rama is programmed entirely with a Java API".

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#120

Post by nathanmarz »

NoraCodes wrote: post #2

I would argue that this is not "a Mastodon instance", since it is not running Mastodon - other than that, very very neat work! I'm excited for that "Source Code" link to be live :)

We call it a "Mastodon instance" because we implemented the entire Mastodon API (https://docs.joinmastodon.org/api/). This is in addition to also implementing the ActivityPub API which Mastodon also implements for federation.
Post reply on HN