Show HN: On-disk B+ tree for Python 3

Nice, I'm a big user of various data stores for scientific work. How does this compare to LMDB (http://www.lmdb.tech/doc/), which uses B-trees (and has a Python interface)?

Re: Show HN: On-disk B+ tree for Python 3

#5

Post by halayli » Sun, Jan 21, 2018, 8:01 PM UTC

What's wrong with LMDB?

Re: Show HN: On-disk B+ tree for Python 3

#6

Post by IvyMike » Sun, Jan 21, 2018, 8:02 PM UTC

nicolaslem wrote: post #3

antman wrote: post #2

Cannot delete items yet

Yes, it's a work in progress, see the `remove` branch.

That is a very dangerous name for a branch.

Re: Show HN: On-disk B+ tree for Python 3

#7

Post by liopleurodon » Sun, Jan 21, 2018, 8:04 PM UTC

IvyMike wrote: post #6

nicolaslem wrote: post #3

Earlier quoted context omitted.

Yes, it's a work in progress, see the `remove` branch.

That is a very dangerous name for a branch.

lol, truth

Re: Show HN: On-disk B+ tree for Python 3

#8

Post by nicolaslem » Sun, Jan 21, 2018, 8:13 PM UTC

dimatura wrote: post #4

Nice, I'm a big user of various data stores for scientific work. How does this compare to LMDB ( http://www.lmdb.tech/doc/ ), which uses B-trees (and has a Python interface)?

I haven't done any benchmark yet but I expect my implementation to be at least an order of magnitude slower.

I've found my implementation to be CPU intensive: creating Python objects from the raw pages is expensive. That's why bulk inserts and iterations are much faster than insert/get in a loop.

Re: Show HN: On-disk B+ tree for Python 3

#9

Post by zerokernel » Sun, Jan 21, 2018, 10:36 PM UTC

https://pypi.python.org/pypi/BTrees

Re: Show HN: On-disk B+ tree for Python 3

#10

Post by laurencerowe » Sun, Jan 21, 2018, 10:38 PM UTC

ZODB has a mature B+Tree implementation for on disk use in the BTrees package. https://pypi.python.org/pypi/BTrees