Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rama is a testament to the power of Clojure

blog.redplanetlabs.com

Post reply on HN
Print view View original

11–20 of 76 posts

Re: Rama is a testament to the power of Clojure

#11

Post by nathanmarz »

finnh wrote: post #8
nathanmarz wrote: post #3

Earlier quoted context omitted.

Check out twitter-scale-mastodon, which is an implementation of Mastodon's backend from scratch that scales to Twitter scale. It's more than 40% less code than Mastodon's backend and 100x less code than Twitter wrote to build the equivalent. https://github.com/redplanetlabs/twitter-scale-mastodon https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...

> These numbers are a bit better than Twitter’s numbers. Because of how unbalanced the social graph is, getting performance this good and this reliable is not easy. For example, when someone with 20M followers posts a status, that creates a huge burst of load which could delay other statuses from fanning out. How we handled this is described more below. Huh. I haven't written myself a twitter, but I've always assumed…

The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the materialized timeline rather than a ton of queries for everyone you follow.

Re: Rama is a testament to the power of Clojure

#12

Post by geodel »

metadat wrote: post #2

> Rama can build end-to-end backends at any scale on its own in a tiny fraction of the code. At its core is a new programming language implementing a new programming paradigm.. Grand claims, but where are the super powerful demo apps? The getting started docs are a trickle of info.. https://redplanetlabs.com/docs/~/operating-rama.html#_access... The "demo gallery" has a few cherry picked examples, but I don't see how…

Well there is whole bank transfer implementation there and thoroughly commented too. Some entrepreneur can take this and run with it to create next generation banking platform.

Re: Rama is a testament to the power of Clojure

#14

Post by epolanski »

> Lisps have great control over what happens at compile-time, which lets you do incredible things.

> Lisp programmers have struggled ever since it was invented to explain why this is so powerful and why this has a major impact on simplifying software development

I've written some Common Lisp, Scheme, Racket.

I like them.

But what op defines as a feature is actually what kills all those lisps but Clojure where macro abuse is rare.

Everybody implementing their abstractions, every library implementing their own language, I like those features, macros are fun, but it just doesn't scale neither in open source and even less at work.

Haskell to some extent too suffers the same issue, simple Haskell is nowadays a dead project, but every single project has different language extensions, syntax, etc..

I feel like Lisp and Haskell attract people that love programming more than shipping code.

Which is why at the end of the day, php has more killer software than all those languages combined.

Re: Rama is a testament to the power of Clojure

#16

Post by cactusfrog »

metadat wrote: post #2

> Rama can build end-to-end backends at any scale on its own in a tiny fraction of the code. At its core is a new programming language implementing a new programming paradigm.. Grand claims, but where are the super powerful demo apps? The getting started docs are a trickle of info.. https://redplanetlabs.com/docs/~/operating-rama.html#_access... The "demo gallery" has a few cherry picked examples, but I don't see how…

The Clojure community is extremely cult like. I worked with a dev who tried to push Datomic, and despite the huge flaws with his system, and the fact that it was closed source, he would continuously tout how much better it was than a relational database. I don’t believe the hype.

Re: Rama is a testament to the power of Clojure

#18

Post by finnh »

nathanmarz wrote: post #11
finnh wrote: post #8

Earlier quoted context omitted.

> These numbers are a bit better than Twitter’s numbers. Because of how unbalanced the social graph is, getting performance this good and this reliable is not easy. For example, when someone with 20M followers posts a status, that creates a huge burst of load which could delay other statuses from fanning out. How we handled this is described more below. Huh. I haven't written myself a twitter, but I've always assumed…

The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the materialized timeline rather than a ton of queries for everyone you follow.

Good to learn, thanks. "Everyone you follow" isn't quite what I was saying, but I also don't use twitter so I'm doubly in the dark here.

I'm surprised that someone with, say, a Twitter problem and tens of millions of followers - the two seem to go hand-in-hand - drives tens of millions of writes every time they post. But there you go, learned something today.

Re: Rama is a testament to the power of Clojure

#19

Post by nathanmarz »

finnh wrote: post #18
nathanmarz wrote: post #11

Earlier quoted context omitted.

The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the materialized timeline rather than a ton of queries for everyone you follow.

Good to learn, thanks. "Everyone you follow" isn't quite what I was saying, but I also don't use twitter so I'm doubly in the dark here. I'm surprised that someone with, say, a Twitter problem and tens of millions of followers - the two seem to go hand-in-hand - drives tens of millions of writes every time they post. But there you go, learned something today.

Back in the days of the fail whale, Twitter would go down when Justin Bieber tweeted because their system couldn't handle the load.

Re: Rama is a testament to the power of Clojure

#20

Post by fizx »

finnh wrote: post #18
nathanmarz wrote: post #11

Earlier quoted context omitted.

The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the materialized timeline rather than a ton of queries for everyone you follow.

Good to learn, thanks. "Everyone you follow" isn't quite what I was saying, but I also don't use twitter so I'm doubly in the dark here. I'm surprised that someone with, say, a Twitter problem and tens of millions of followers - the two seem to go hand-in-hand - drives tens of millions of writes every time they post. But there you go, learned something today.

I was tech lead on that subsystem for a little while in 2010. A lot of smart people thought about the hybrid approach, either by using the search index to drive the timeline or building a custom ring-buffer-based index of all tweets. Ultimately two systems are harder to maintain than one, custom indices are hard, and the low-complexity approach dominated a higher-performance approach.

Also, contrary to popular opinion, we didn't go down when Justin Bieber tweeted, but we did have elevated error rates when large quantities of Justin Bieber followings put pressure on the MySQL row lock of his following count. In retrospect, lock striping would have helped, but the migration would have been horrific.

Post reply on HN