31–40 of 82 posts
What I would like to know is, is there support accessing values by nth-largest key? I had to roll my own b-tree library specifically to get this feature since nothing out-there had it.
Then lookup the n-th largest key and then pull the values?
eg. SELECT value from key_value_pairs order by size(key) LIMIT N
This looks like it has similar goals to bitcask. The tradeoffs and differences would be nice to know.
leveldb is a persistent ordered map; bitcask is a persistent hash table (no ordered iteration). bitcask stores a fixed size record in memory for every key. So for databases with large number of keys, it may use too much memory for some applications. bitcask can guarantee at most one disk seek per lookup I think. leveldb may have to do a small handful of disk seeks. To clarify, leveldb stores data in a sequence of lev…
Earlier quoted context omitted.
One of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usually. I haven't done a careful head-to-head comparison, but it wouldn't surprise me if it was somewhat faster than leveldb for many workloads. Plus TokyoCabinet is more mature, has matching server code etc. and may therefore be a better fit for many projects. Howev…
Did you consider fractal trees? http://tokutek.com/presentations/bender-Scalperf-9-09.pdf
This looks like it has similar goals to bitcask. The tradeoffs and differences would be nice to know.
leveldb is a persistent ordered map; bitcask is a persistent hash table (no ordered iteration). bitcask stores a fixed size record in memory for every key. So for databases with large number of keys, it may use too much memory for some applications. bitcask can guarantee at most one disk seek per lookup I think. leveldb may have to do a small handful of disk seeks. To clarify, leveldb stores data in a sequence of lev…
Earlier quoted context omitted.
Based on their numbers it looks slower than TokyoCabinet but still much faster than BerkleyDB. Would need to design a head to head benchmark. I don't think Redis is a good comparison as that's an in-memory database so better suited for the 10% of hot data you'd need to cache. Whereas disk stores like TokyoCabinet and LevelDB would be great for storing the other 90-100%. If your use case involves a large dataset and y…
One of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usually. I haven't done a careful head-to-head comparison, but it wouldn't surprise me if it was somewhat faster than leveldb for many workloads. Plus TokyoCabinet is more mature, has matching server code etc. and may therefore be a better fit for many projects. Howev…
Anybody know what hardware those benchmarks were on? Is it actually durable? They mention problems with writeback caching, but are they at least durable from the OS perspective? How well does it scale?
It is durable from an OS perspective. I started typing in a long explanation, but I think the comment for the "sync" field in the options.h file captures things well: struct WriteOptions { // If true, the write will be flushed from the operating system // buffer cache (by calling WritableFile::Sync()) before the write // is considered complete. If this flag is true, writes will be // slower. // // If this flag is fal…
Earlier quoted context omitted.
leveldb is a persistent ordered map; bitcask is a persistent hash table (no ordered iteration). bitcask stores a fixed size record in memory for every key. So for databases with large number of keys, it may use too much memory for some applications. bitcask can guarantee at most one disk seek per lookup I think. leveldb may have to do a small handful of disk seeks. To clarify, leveldb stores data in a sequence of lev…
Also worth noting: bitcask is written in erlang and leveldb in c++.
Earlier quoted context omitted.
It is durable from an OS perspective. I started typing in a long explanation, but I think the comment for the "sync" field in the options.h file captures things well: struct WriteOptions { // If true, the write will be flushed from the operating system // buffer cache (by calling WritableFile::Sync()) before the write // is considered complete. If this flag is true, writes will be // slower. // // If this flag is fal…
Writes being lost means what, a trashed file? Or merely an incomplete one?
What I would like to know is, is there support accessing values by nth-largest key? I had to roll my own b-tree library specifically to get this feature since nothing out-there had it.
Why not index size(key)? Then lookup the n-th largest key and then pull the values? eg. SELECT value from key_value_pairs order by size(key) LIMIT N
How would you use this approach to find say, the 195687th largest element in O(log(n)) time?
Edit: However, I do know I can write a SQL query to find this - an ordinary index plus limit should do this. It's just such an approach gets really awkward and so unpredictable when you are dealing with a lot of distinct columns and tables - why I implemented this with key-value DB (a custom index on top of Kyoto Cabinet).