181–190 of 376 posts
Earlier quoted context omitted.
did you ever consider starting from something already technically performant like pleroma or misskey?
Well, we didn't start from anything as we implemented this completely from scratch. I believe Mastodon is much more widely used than those so it seemed like a better target for this.
Earlier quoted context omitted.
What do you mean? Every post I do shows up instantly. Reloading the page from scratch can be slow due to Soapbox doing a lot of stuff asynchronously from scratch (Soapbox is the open-source Mastodon interface that we're using to serve the frontend). https://soapbox.pub/
I think the concern is will this still be true if Mastodon reaches Twitter scale?
Rama's built-in telemetry provides the information you need to know when it's time to scale.
Earlier quoted context omitted.
So Rama-powered apps need to be written in Java? Or will any JVM language work? And the Rama core will remain closed-source? That part seems like the toughest sell of all, at a time when the vast majority of developer tooling and backends are open source or at the very least source-available.
Any JVM language should work. We've built modules with Clojure. We're keeping it closed-source for now.
Rama sounds interesting to me for my 'next big project', but I'd not even consider building it on top of a closed core. I think this is a pretty common sentiment in these circles.
I understand building an OSS business is not easy either. But perhaps there is some middle of the road that you can walk?
- A contractual obligation to open source all (now current) code a couple of years in the future? - Or an almost-OSS license that makes life difficult for competing cloud providers, like https://www.mongodb.com/licensing/server-side-public-license... ?
Earlier quoted context omitted.
The short answer is write-through cache. You write the update directly to the cache closest to the user and into the eventually consistent queue. We did this at reddit. When you make a comment the HTML is rendered and put straight into the cache, and the raw text is put into the queue to go into the database. Same with votes. I suspect they do this client side now, which is now the closest cache to the user, but back…
In Nathan Marz's (the article author) book, Big Data, he describes this and calls it the Speed Layer. I haven't fully finished the article yet, but the components it's describing seem to be equivalent to what he calls the Batch Layer and the Serving Layer in his book. But I'm kind of getting the impression this works without any speed layer and is expected to be fast enough as-is.
Rama is not batch-based. That is, PStates are not materialized by recomputing from scratch. They're incrementally updated either with stream or microbatch processing. But PStates can be recomputed from the source data on depots if needed.
This is what they've been hyping on Twitter for a week? FWIW, why hype at all? Why "We'll more in a week. Then more in two weeks." Show the code today!
Considering the length and amount of detail in this blog post, I understand why they would need another week to get the code ready (assuming there will be more docs)
Saying this is 100 or even a million times cheaper is like saying taking a picture of Sistine chapel and printing out copies is a trillion times cheaper than making it originally.
Many of us on this site could make a number of products very efficiently and cheaply given a static and fixed set of requirements as well as an existing implementation for reference.
That being said it was a very detailed post, so kudos for that, but it’s far too vague to be actionable. Why not just release the code and post simultaneously instead of just bragging about how little code was required?
Looks amazing and incredibly smart. But I found the LOC and implementation time comparisons to Twitter and Threads very disingenuous. It makes me wonder what other wool will be pulled over our eyes with Rama in future (or important real world details missed / future footguns). Still super impressive. Reminds me of when I discovered Elixir while building a social-ish music discovery app. Switching the backend from Rai…
+1 the comparisons are not great. How much engineer-hours did it take to build Rama itself? The numbers they got for Twitter likely include the time it took to build their infrastructure, common libraries (like finagle,…)
Earlier quoted context omitted.
Their point is to show off the power of Rama, I.e. it is possible to build such applications on top with little work.
Exactly, why are so many people missing this point. It's not "we built a narrow, tedious framework for knocking off Twitter clones", it's "We built a platform that turns data processing on its head and look in a couple of months you can clone Twitter just imagine what YOU can do with this." I see parallels though to Datomic, where they turned the database inside out, co-located the app logic and data and indexes, etc…
It’s a massive ask, even if the platform was 100x better, for all developers to give up every programming language and database they’ve ever used to depend on a startups closed source platform for all functionality. It’s hard enough trusting Google or Amazons cloud offerings won’t change. It seems that’s what they’re proposing right? What am I missing?
We're actually not asking anyone to give up anything. First off, it has a simple integration API (which you'll be able to see the details of next week) that allows it to seamlessly integrate with any other backend tool (databases, monitoring systems, queues, etc.). So Rama can be incrementally introduced into any existing architecture. Second, Rama has a pure Java API and is not a bespoke language. So no new language…
Big if true (and if the opposite, of incrementally removing it also works). There have been similar platform efforts in past, such as https://news.ycombinator.com/item?id=20985429 . For that one, the "massive ask to give up every programming language and database they’ve ever used to depend on a startups closed source platform" seems like the biggest hindrance to adoption.
> ...10k lines of code. This is 100x less code than the ~1M lines Twitter I wish I didn't see this comparison, which is not fair at all. Everyone in their right mind understands that the number of features is much less, that's why you have 10k lines. Add large-scale distributed live video support at the top of that, and you won't get any close to 10k lines. It's only one of many many examples. I really wish you compa…
Granted, a decentralized platform would eliminate some of those, just by being decentralized