What's a Riak and is there really no better link for this news?
Open Riak – open, modern Riak fork
51–60 of 76 posts
Re: Open Riak – open, modern Riak fork
#52This really hits home and makes me happy to see on the HN front page.
Nearly 10 years later and I still consider my time working on Riak at Basho the highlight of my career.
After leaving, my original plan was to found "Basho 2.0" after my non-compete expired. But, unexpected personal/family hardships in 2015-2018 made big-tech money the better choice for awhile, and Cloud/competitors continued to chip away at the market.
Often stil regret not taking that path.
But, happy to see technology I'm very fond of still living on and providing value to the world.
Re: Open Riak – open, modern Riak fork
#53Earlier quoted context omitted.
Metastability is an under-rated system property for databases and systems software, in general.
What does metastability mean in this context? I've only seen it used to mean "appears stable but not actually stable", eg systems that resist small perturbations but never return to nominal after bigger disturbance (like cold boot). Did you mean "stability"?
Marc Brooker’s blog on the topic is good: https://brooker.co.za/blog/2021/05/24/metastable.html
Re: Open Riak – open, modern Riak fork
#54A question possibly answered elsewhere, but did openriak include only Risk KV, or also other projects like CS & TS?
Re: Open Riak – open, modern Riak fork
#55Earlier quoted context omitted.
We were working on ways of making it easier (such as CRDTs to reduce the amount of work developers had to do to leverage eventual consistency), but these were pretty challenging problems to solve. One of our biggest disappointments: we had plans to add a way to enforce strong consistency leveraging (IIRC) something akin to multi-paxos, but couldn't get it to work.
TBH, we shipped fully working strong consistency in 2014. It just had a limited feature set, was disabled by default, and was never promoted/marketed since it didn't fit the direction the new CEO/CTO was pushing. The engineering exodus around that time sorta killed the project though, and we never were able to do the big follow-up work to make it really shine. (Disclaimer: Former Basho Principal Engineer, primary aut…
There were customers happily using strong consistency in production, but somehow the idea that it wasn’t “finished” kept getting repeated over and over by management. I was well on my way to solving the biggest rough edge (tombstone reaping in SC buckets) but then I got pulled off to work on the infamous “data platform” and never got to finish that work :-(
Re: Open Riak – open, modern Riak fork
#56I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.
Companies would rather use something like dynamodb than self-host riak. You get an army of Amazon code monkeys to help you if something goes wrong, and it's a click away.
Re: Open Riak – open, modern Riak fork
#57Earlier quoted context omitted.
We were working on ways of making it easier (such as CRDTs to reduce the amount of work developers had to do to leverage eventual consistency), but these were pretty challenging problems to solve. One of our biggest disappointments: we had plans to add a way to enforce strong consistency leveraging (IIRC) something akin to multi-paxos, but couldn't get it to work.
TBH, we shipped fully working strong consistency in 2014. It just had a limited feature set, was disabled by default, and was never promoted/marketed since it didn't fit the direction the new CEO/CTO was pushing. The engineering exodus around that time sorta killed the project though, and we never were able to do the big follow-up work to make it really shine. (Disclaimer: Former Basho Principal Engineer, primary aut…
I'm not sure though for how much longer it will continue to make sense for the project as-is to continue to roll riak_ensemble forward as part of future releases. As there are no contributors who have direct direct experience or knowledge of using it in production, so it is hard to claim it as being a supported part of the product in any real sense.
I apologise if we do eventually cut it. Having worked through the code when chasing unstable tests, I developed an appreciation for the quality of the work.
Re: Open Riak – open, modern Riak fork
#58I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.
> I've never met an engineering team that used Riak
I was part of a recent cloud migration. Part of on-prem (though unfortunately not migrated by my team) were this very first Riak Cluster I saw in production.
The engineering team used it as "kind of S3" for images, with 3 to 5 PHP scripts providing an interface to Riak and imageMagic. It seemed to me like a good abstraction and I think the migration to S3 was mostly painless.
Other than that I only had contact with Riak at university around 15 years ago, when we tested cluster setups of several NoSQL databases and tried to manually introduce faults to see if they could heal. Riak passed our test at that time, MongoDB didn't.
Re: Open Riak – open, modern Riak fork
#59I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.
Speaking as a former tech evangelist/engineer at Basho, there were a few significant challenges. Riak is horribly unfriendly as a database: no SQL, it exposes eventual consistency directly to the developer, it’s relatively slow, and Erlang is a fairly unusual language. While you can run Riak on a single server, you’d have to really want to. Its strength is the ability to scale massively, but not many projects need th…
I joined a company that had some investment in Basho and managed to sell Riak as the data store for a large client. It never really worked out, not enough of the SREs had be properly trained on Riak, the developers hated it because getting help and support was somewhat difficult, especially in an emergency.
In the end more and more data was offloaded to MariaDB, until one day the last remaining data couldn't justify the cost of the Riak cluster. I think we swapped out an eight node Riak cluster for two largish MariaDB database (one being a hot-standby).
For one of the other clients it was the exact same scenario, only we had been contracted in to help run the Riak cluster, which we didn't do well. Once they had migrate of it, to Oracle I think, the client left.
To me it always felt like it was just the wrong tool for that particular job. Someone really wanted to be able to jump on the NoSQL hype and sell something. They picked Riak, because it honestly looked really good, and probably was, compared to MongoDB, CouchDB or whatever else happened to float around at the time. It just wasn't the right tool for the problems it was applied to.
Re: Open Riak – open, modern Riak fork
#60Riak has been maintained through the post-basho years by engineers at some of its larger customers (disclaimer - including myself). The focus has been on trying to improve the stability of the database when subject to complex failure scenarios under stressful load, with minimal need for urgent operator intervention. The focus has been on keeping those existing operators happy rather than seeking out new users. Evolut…
Metastability is an under-rated system property for databases and systems software, in general.
I'm not aware of any definition of metastability that would be a desirable property of a system. Vaguely speaking, in my understanding metastability in physics, chemistry and electronics refers to a state that has not yet fully settled and the final state is therefore undeterminable / random.
Could you elaborate your understanding of the term? I can see it related to concepts like eventual consistency, just unclear of how it would be considered a positive characteristic. I'd have thought that a deterministic outcome would be important in a database system?