Live data from Hacker News

We reduced the cost of building Mastodon at Twitter-scale by 100x

blog.redplanetlabs.com

361–370 of 376 posts

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#361

Thank you for sharing this! I am actually really impressed. Well done! Good work! There's lots of interesting lessons and knowledge in the design of this platform. I also like how you've decided to use Java as your API rather than Clojure. I hope you're not discouraged by HN's reaction to your hard work. Don't be discouraged!

This isn't my first time releasing a big project, so I'm used to the HN knee-jerk reactions :) Thanks for the kind words.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#362
Really nice ideas here. The crucial advantage is having the storage+computation run as close as possible, which is a big advantage over a regular DB+app backend.

But I won't ever consider investing in it unless it's some form of open-source. It's too much of a risk to have a closed-source core.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#363
post #207

Earlier quoted context omitted.

In a typical architecture, the DB stores data, and the backend calls the DB to make updates and compile views. Here, the "views" are defined formally (the P-states), and incrementally, automatically updated when the underlying data changes. Example problem: Get a list of accounts that follow account 1306 "Classic architecture": - Naive approach. Search through all accounts follow lists for "1306". Super slow, scales…

I read their post and honestly it’s not really that much different than just materialized views in a regular database plus async jobs to do the long running tasks. It’s a ridiculous amount of fluff to describe that. Not to mention it’s proprietary and only supports the JVM and doesn’t integrate with the tons of tooling designed about RDBMS unless you stream everything to them, defeating the purpose. What really irks…

> I read their post and honestly it’s not really that much different than just materialized views in a regular database plus async jobs to do the long running tasks.

How about you go and implement a Mastodon server to their level of feature parity, and tell us how much effort and how many lines of code it takes?

I really don't appreciate this kind of fluffy, insubstantial, overly dismissive non-content on HN.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#365
post #330

Earlier quoted context omitted.

Meh. By this definition all libraries expose an "embedded DSL" — their API. I'm honestly not sure this is a useful definition.

Whether you like it or not; internal DSLs became a thing with Ruby back in the day. And these days things like Kotlin also lend themselves pretty well to creating internal DSLs. Java is not ideal for this. Kotlin and Ruby have a few syntax features that make it very easy.

What's the distinguishing difference between a regular library and an embedded DSL?

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#366
post #365

Earlier quoted context omitted.

Whether you like it or not; internal DSLs became a thing with Ruby back in the day. And these days things like Kotlin also lend themselves pretty well to creating internal DSLs. Java is not ideal for this. Kotlin and Ruby have a few syntax features that make it very easy.

What's the distinguishing difference between a regular library and an embedded DSL?

The difference is making an effort to expose the features of the library via a syntactically friendly way. Most Kotlin libraries indeed have nice APIs that are effectively mini DSLs.

If you need an example, Kotlin uses a nice internal DSL for HTML where you write things like

  Div {
     P { 
        +"Hello world"
     }
  }

This is valid Kotlin that happens to construct a dom tree. There is no magic going on here; just usage of a few syntax features of the language. Div and p here are normal functions that take a receiver block as the last parameter that receive an instance of the element they are creating. The + in front of the string is function with a header like this in the implementation of that element.

   operator fun String.unaryPlus()

The same code in Java would be a lot messier because of the lack of syntactical support for this. You basically end up with a lot of method chaining, semicolumns and their convoluted lamda syntax. The article has a few examples that would look a lot cleaner if you'd rewrite them in Kotlin.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#367

Earlier quoted context omitted.

Are you sure about that. With things like twitter, the ui is not the hard part. Things like moderation are the secret sauce. All the corner cases and support for devopsy stuff likely account for a lot. Routing to specific instances for celebrities and such.

Nathan worked at Twitter so while he might be wrong, I don't think it's reasonable to assume he's just naive http://nathanmarz.com/blog/leaving-twitter.html .

I misread the parent.

I originally read it as it's easy to clone twitter.

My response is it's easy enough to build a micro-blogging platform/service. it's all the other shit like moderation, regulatory/legal compliance, making a profit, keeping advertisers happy, etc that's hard.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#368
post #321

Earlier quoted context omitted.

Yeah, a very small amount so that clicking the upvote button does not need to reload the whole page

Upvoting does work javascript disabled, it upvotes and reloads the page. It approximately keeps your position by jumping the message anchor.

I know it works with js disabled as well. But with js enabled they send just the upvote instead of reloading the whole page. Saving network traffic.

Unfortunately the js used is very minimal. And if the upvote fails, there is no graphical indication that it failed to upvote.

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#369
post #207

I do C++ backend work in a non-web industry and this entire post is Greek to me. Even though this is targeted at developers, you need a better pitch. I get "we did this 100x faster" but the obvious followup question is "how" but then the answer seems to be a ton of flow diagrams with way too many nodes that tell me approximately nothing and some handwaving about something called P-States that are basically defined to…

In a typical architecture, the DB stores data, and the backend calls the DB to make updates and compile views. Here, the "views" are defined formally (the P-states), and incrementally, automatically updated when the underlying data changes. Example problem: Get a list of accounts that follow account 1306 "Classic architecture": - Naive approach. Search through all accounts follow lists for "1306". Super slow, scales…

This is a problem that is very similar to the one solved by Differential Data Flow and implemented by https://github.com/MaterializeInc/materialize

Have you considered using that?

Re: We reduced the cost of building Mastodon at Twitter-scale by 100x

#370

> ...10k lines of code. This is 100x less code than the ~1M lines Twitter I wish I didn't see this comparison, which is not fair at all. Everyone in their right mind understands that the number of features is much less, that's why you have 10k lines. Add large-scale distributed live video support at the top of that, and you won't get any close to 10k lines. It's only one of many many examples. I really wish you compa…

> Add large-scale distributed live video support at the top of that, and you won't get any close to 10k lines. But Twitter isn't, and was never, about live video support: this is pure feature creep and that's how you get headcount inflation and a company that can be run for 17 years without making profit (AKA terrible business). > When I worked in Statuspage, we had support of 50-100k requests per second Having serve…

> Having served 150kqps in the past as part of a very small team (3 back-end eng.), this isn't necessarily as big of a deal

I would love to see a blog post about your experience!

Post reply on HN