Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
1–10 of 19 posts
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#2In the '90s, two brothers, friends of mine, had built a series of commercial software using a similar implementation - double-entry accounting, a construction project evaluator, and several others. When everybody was using dBase and FoxPro at the time and struggling, the speed of development and the low hardware requirement of their products was unmatched.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#3https://rabbitmq.github.io/khepri/ has a bit more information on why you might want to use this, from what I can understand. It's a bit over my head. I guess its sort of simpler to manage a bunch of data on a disk vs a regular db (when not considering that just a bunch of data on disk), mostly around network issues?
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#4Khepri is an interesting alternative to Neo4j for a program I wrote to store tree data a few years ago. Of course, I'd have to try it out with actual data, but it seems performant in the few minutes I've been fiddling with it.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#5My understanding after spending a few minutes reading through the project web page and stack overflow listings about RabbitMQ clustering problems is as follows.
Khepri is a project intended to replace Mnesia for replicated, clustered distributed systems written in Erlang/Elixir such as RabbitMQ. The primary reason this project started was to address shortcomings of Mnesia with regards to "network partitions", where cluster nodes are running on unreliable networks and network failures happen.
Caveat emptor! The project README still refers to Khepri as an alpha product, so I assume that RabbitMQ continues to use some kind of customized Mnesia for its production system. Unfortunately, in the absence of any significant system using Khepri, should you decide to adopt it you will be on the bleeding edge and will have scars for using it, at best, and a complete project failure at worst. Distributed systems problems are very hard problems to solve.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#6This is only tangentially related to Khepri, but I'm curious why the STEM community started treating the word "data" like a plural noun, e.g., "Data ARE stored in a tree structure"?
I know in Latin, "data" is plural and "datum" is singular, but we aren't speaking Latin, and even if we were, we're not doing so consistently. For example, we don't say, "The meeting's agenda ARE up to date," even though "agenda" is plural and "agendum" is singular. Instead, we adapt the word "agenda" to our grammar and use it as a singular collection, "The meeting's agenda IS up to date," similar to the way we say, "The population IS growing."
To me, saying "Data are..." is pretentious, like when people use the word "an" before words that start with a consonant.
EDIT: Fixed a capitalization typo.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#7This is only tangentially related to Khepri, but I'm curious why the STEM community started treating the word "data" like a plural noun, e.g., "Data ARE stored in a tree structure"? I know in Latin, "data" is plural and "datum" is singular, but we aren't speaking Latin, and even if we were, we're not doing so consistently. For example, we don't say, "The meeting's agenda ARE up to date," even though "agenda" is plura…
"Started"? That both forms are used, somewhat depending on field, is not really a new thing.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#8This is only tangentially related to Khepri, but I'm curious why the STEM community started treating the word "data" like a plural noun, e.g., "Data ARE stored in a tree structure"? I know in Latin, "data" is plural and "datum" is singular, but we aren't speaking Latin, and even if we were, we're not doing so consistently. For example, we don't say, "The meeting's agenda ARE up to date," even though "agenda" is plura…
"Started"? That both forms are used, somewhat depending on field, is not really a new thing.
It's relatively new to me. When I was in college, books like "The Future of Data Analysis (JW Tukey, 1962) used "data is." I believe all of my CS textbooks treated the word "data" as a singular collection back then. I've only noticed the ubiquitousness of "data are" in engineering books since the turn of the century.
It could be a false memory though, which is why I'm asking.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#9Earlier quoted context omitted.
"Started"? That both forms are used, somewhat depending on field, is not really a new thing.
It's relatively new to me. When I was in college, books like "The Future of Data Analysis (JW Tukey, 1962) used "data is." I believe all of my CS textbooks treated the word "data" as a singular collection back then. I've only noticed the ubiquitousness of "data are" in engineering books since the turn of the century. It could be a false memory though, which is why I'm asking.
My impression from styleguides etc is that the singular form is the newer one, and in some cases only somewhat begrudgingly accepted. E.g. recommendations that it is acceptable to use singular when writing for a "general audience", not in "formal or scientific writing", others referring to plural as "still in common use" (but singular taking over). But maybe there's been indeed a return to it in this specific field? not sure.
Re: Khepri is a tree-like replicated on-disk database library for Erlang and Elixir
#10This is interesting in that it uses Raft. I had a go at implementing a toy Raft implementation and I think testing my particular implementation properly would be pretty hard to do without TLA or automated proofing algorithms. You could also use Jepsen.
I maintained a clustered RabbitMQ cluster and used Chef to coordinate upgrades. Uprading Erlang AND RabbitMQ at the same time wasn't very enjoyable since it had the potential to go wrong in production. Fortunately it went right and nobody noticed.
If RabbitMQ can solve the partition problem with Khepri that would be great. The userguide tells how to configure RabbitMQ to handle splits.
https://www.rabbitmq.com/partitions.html
I think we used pause_minority which sacrifices availability for consistency.