21–24 of 24 posts
I've been staring to write a paper on bcache/bcachefs's b-tree design, which is also a sort of combination b-tree/trie (it uses binary trees in eytzinger layout). Pretty cool to see the same idea coming up in multiple places. Running index-microbenchmarks now to see which is faster :)
Single threaded random lookups: 1.4M/sec for bcachefs, 2M/sec for masstree.
I've been staring to write a paper on bcache/bcachefs's b-tree design, which is also a sort of combination b-tree/trie (it uses binary trees in eytzinger layout). Pretty cool to see the same idea coming up in multiple places. Running index-microbenchmarks now to see which is faster :)
Earlier quoted context omitted.
Cool, looking forward. Please put Judy in as well :) Seriously, the 17 year old judy is still pretty good, despite it's lack of use of cool vector load / compare / etc instructions for quickly traversing tree nodes that are too small for a full radix search (ART does "simultaneous search", i.e. compares to all stored keys in a single instruction, while judy afaik runs a linear search). It would be pretty cool if some…
I could either figure out how Judy works, or review another three papers :)
My super high-level partial understanding of Judy is the following:
Start with ART, which is pretty simple. Then do the obvious improvements: First, we want fast access to "element number N in sorted order", i.e. you also store number of descendants. Next, you do key compression: Storing the portion of the key that can be reconstructed from the tree traversal is silly. Next: an 8-bit tag for signaling one out of 4 node types? Are you, like, filthy rich? More node types it is. Next: Spending 64 bit for a node pointer? Are you crazy? Put the type-tag into the parents pointer (earlier resolution of the branch!), and use a custom allocator to get smaller pointers (you allocate big segments and your node pointers are offsets).
You end up with an unintelligible monstrosity. Give it a cute name, forget about SIMD because it is 2001, and you end up with something like Judy.
So what are the advantages over adaptive radix trees or good old judy-dict/array? Apart from judy being too damn complicated, and too old to be optimized for vector-compare instructions (I think the fancy hand-coded x86 vector-comparisons are the main reason for ART being competitive with judy, considering that it misses at least key compression and the clever allocator, and that ART is not as optimized for using eve…
Here's a recent comparison of Masstree to ART: https://twitter.com/andy_pavlo/status/986647389820747776?s=2... ART looks to be better in most cases. It's on my list of K-V stores to review: https://www.the-paper-trail.org/page/reading-list/
Something gets bonus points for not needing other libraries or static data, so I could put it in a kernel, but usually I don't really need this.
For example, Judy is LGPL. Darn. That alone is trouble, but Judy makes it more questionable due to non-trivial code in *.h files.
Any good options?