Live data from Hacker News

Open Riak – open, modern Riak fork

github.com

21–30 of 76 posts

Re: Open Riak – open, modern Riak fork

#21
post #5

Who would this be for in 2024? I remember evaluating Riak back in 2011 or so for an analytics solution, but ended up going with a more traditional OLAP database that was a much better option. It's hard for me to imagine where Riak would be a good option given how many choices we have today for various data stores.

It is a fault tolerant massively scalable key value store capable of handling hundreds of terabytes of data.

What are these options you are thinking of?

The only thing that comes to my mind is Aerospike and possibly ScyllaDB.

Re: Open Riak – open, modern Riak fork

#22
post #7

I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.

I worked on a team that built a massive, high-performance internal service based on Riak. There are many things I learned from that system. Here is the best takeaway I can offer:

It does not matter what your technology is, or how theoretically superior it is. Getting it to actually work well "in production" is a whole separate thing than simply designing it and writing code. When it's a very small system, it will look like it's doing great. As it gets bigger, the seams will start to burst, and you will find out that promises and theory don't always match reality.

In the end, while its aims are great, it takes a whoooooole lot of work to smooth out the bumps in such a system. You need experts in that technology to address bugs in a timely manner. You need developers versed in the system to properly build apps utilizing it. You need competent operators to build, orchestrate, operate and maintain the whole thing.

All of that is made easier by using simple technology that everybody knows, that there's a huge support community for, professional services for, etc. A technology like MySQL or Postgres etc, has the corporate, development, support, etc to make it easy to work with at any scale. A little janky at times, limited, but dependable, predictable, controllable.

A small bespoke system with a small support community and virtually no corporate support is, comparatively, a hell of a lot more difficult/costly to support and harder to make work reliably.

Re: Open Riak – open, modern Riak fork

#24
post #5

Who would this be for in 2024? I remember evaluating Riak back in 2011 or so for an analytics solution, but ended up going with a more traditional OLAP database that was a much better option. It's hard for me to imagine where Riak would be a good option given how many choices we have today for various data stores.

It's realistically for the handful (dozens at most?) of very large Riak implementations where it would be enormously expensive to rewrite the application running on top of it.

For example, the UK NHS Spine messaging system which has been building on Riak for 10 years

https://riak.com/posts/press/nhs-launches-upgraded-it-backbo...

Re: Open Riak – open, modern Riak fork

#25
I led a migration from Mongo to Riak at Shareaholic about 12 years ago: https://www.slideshare.net/slideshow/migrating-to-riak-at-sh...

It was successful at first, but ultimately we traded one set of problems for another (how novel, I know).

In particular, I underestimated the pain of troubleshooting the database itself. Riak was a new product, we were a small team that had never run anything on BEAM, and ultimately we lost too many days debugging and trying to make sense of Erlang stacktraces.

The Basho folks were great, and to this day I appreciate how quickly they fixed a number of bugs for us. But ultimately it wasn't enough -- we found problems faster than they could be patched.

Re: Open Riak – open, modern Riak fork

#26
post #7

I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.

I'm pretty sure Stripe was a heavy user of this for a while. They used it due to their write-heavy system, if I recall.

I fondly remember writing a Go driver for it. Was a good experience: https://github.com/riaken/riaken-core

Re: Open Riak – open, modern Riak fork

#27
post #10
post #8

Earlier quoted context omitted.

Vector clocks are very cool. Having read through how they were initially used in Riak, I was blown away that such an implementation could scale. I guess this is why Cassandra took a different approach?

Vector clocks are certainly cool but fundamentally premised on the idea of having multiple 'live' versions of a value at once. Amazon's original Dynamo paper required conflict resolution at the application level, which is a very strange framework to build applications on. (Notably DynamoDB has moved away from this, I believe to Last Write Wins.) Cassandra takes the latter approach by default as well, I believe.

yes there's that idiosyncrasy, as well as client ideally needing to read the previous clock from the DB before writing an update for that key unless it's ok with the write being viewed as concurrent. Plus the extra memory overhead to store the clocks in the client.

Re: Open Riak – open, modern Riak fork

#28
post #19
post #7

I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.

We used Riak at $dayjob at around 2014-2017 (iirc). I don't exactly remember it fondly. It was slow and unreliable. You could make it freeze/crash with the wrong SOLR query. (I was pretty good at that...)

The SOLR part has now been retired from the last few releases.

Current development has been focused on improving the flexibility of secondary indexes. There was some funky stuff achieved by some users using overloaded 2i terms and distributed processing of regular expressions against those terms - the aim is now to make this more flexible to the modern developer using the language of projected attributes and filter expressions (ala DynamoDB). There's also some active work to both replicate-to and full-sync (i.e. reconcile with) external OpenSearch clusters.

The primary goal for OpenRiak is stability under load/failure as a K/V store - so the ultra-flexibility of in-built SOLR querying has been sacrificed in the move towards that aim. Anything that can do harm is to be offloaded or constrained.

Re: Open Riak – open, modern Riak fork

#29
post #7

I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.

Speaking as a former tech evangelist/engineer at Basho, there were a few significant challenges. Riak is horribly unfriendly as a database: no SQL, it exposes eventual consistency directly to the developer, it’s relatively slow, and Erlang is a fairly unusual language. While you can run Riak on a single server, you’d have to really want to. Its strength is the ability to scale massively, but not many projects need th…

https://howfuckedismydatabase.com/nosql/ this infamous comic is about riak

Re: Open Riak – open, modern Riak fork

#30

Riak has been maintained through the post-basho years by engineers at some of its larger customers (disclaimer - including myself). The focus has been on trying to improve the stability of the database when subject to complex failure scenarios under stressful load, with minimal need for urgent operator intervention. The focus has been on keeping those existing operators happy rather than seeking out new users. Evolut…

Metastability is an under-rated system property for databases and systems software, in general.
Post reply on HN