[deleted]
We reduced the cost of building Mastodon at Twitter-scale by 100x
371–376 of 376 posts
Re: We reduced the cost of building Mastodon at Twitter-scale by 100x
#372Another try of Event Sourcing + CQRS. I thought it was great but after so many years it's still out of main stream. Lack of an integrated platform may contribute to, but can't explain everything.
I guess most people can't accept things which is fundamentally harder in such architecture than normal ones.
Re: We reduced the cost of building Mastodon at Twitter-scale by 100x
#373Earlier quoted context omitted.
"Rama is programmed entirely with a Java API".
The customer API in Java, and the implementation of that API is in Clojure.
Does a possibility of a Python API figure anywhere in your roadmap? Or am I missing the point here? Seems like it would be a smart choice to have one on the statically-typed side, and one on dynamically-typed side?
Re: We reduced the cost of building Mastodon at Twitter-scale by 100x
#374This is very interesting.
I saw the Twitter post first and the blog next. The premise is compelling but it's been a promise made to the data and software world for decades together.
The architecture and the core primitives are something that we agree with a lot. Use cases and business value are a whole different ballgame.
We have invested the past 5 years at InfinyOn building Fluvio our open source rust implementation of core event streaming primitives which is implementing this architecture to orchestrate data as efficiently as computationally possible today. I am happy to see this project as an effort in the same direction.
Re: We reduced the cost of building Mastodon at Twitter-scale by 100x
#375I appreciate the inversion/melding of the data model and compute. Im curious to know your perspective on two parts: How would multitenancy fit in to rama? Using your mastodon example, providing “hosted mastodon instances as a service”, where you _also_ allow for data governance, per customer encryption at rest, user IDP support, etc. Is it multiple single tenant rama deployments, running independent customers? Multit…
For now it's going to be on-prem, so each user will just have their own cluster. Things like E2E encryption are pretty easy to implement on top of Rama's existing primitives (there was a good question about this on the rama-user group yesterday https://groups.google.com/u/1/g/rama-user/c/jj-ILcoMjtk ). We'll likely have a fully managed cloud version in the future. Riak was good technology at the time, but it was real…
The big question is whether we really want more centralised cloud applications of this sort in the AI future, with its potentially exacerbated risks of data breaches, identity theft, spam, etc.
Having a "Rama" for local-first, truly distributed (Solid Project style), self-sovereign-identity based apps would be differentially better probably.
Re: We reduced the cost of building Mastodon at Twitter-scale by 100x
#376Summarizing, now edited down with some editorializing for clarity: What is it? build web-scale reactive backends with an expressive java dataflow API. Instead of a database you develop your own custom app-specific indexes which are reactive, distributed and durable. It's like event sourcing and materialized views but integrated in a linearly scalable way. > I cannot emphasize enough how much interacting with indexes…
Electric could be an ideal companion to these reactive Queries & Depots to build actual UI, btw
We're thinking and planning for the exact same. Curious.