Hi there! I'm a YC alum (reMail W09) and helped Jeff and Sanjay with LevelDB. Let me know if you have any questions about LevelDB and I'll see if I can help.
How does this compare to other persistent key-value stores such as Membase?
LevelDB: A Fast Persistent Key-Value Store
51–60 of 66 posts
Re: LevelDB: A Fast Persistent Key-Value Store
#52I love the insight about how fast compression (Snappy) is like having faster hard drives.
Re: LevelDB: A Fast Persistent Key-Value Store
#53Earlier quoted context omitted.
That leveldb has been discussed several times on HN in the last two months. I just didn't break out the links from the search UI. Downvoters: links to previous context are generally considered a good thing here.
Oh, then its easier to understand that in this query: http://www.hnsearch.com/search#request/all&q=leveldb&...
Yeah, the search results when I linked them had those results. The index was subsequently updated, so the top hits are now this thread, and this article. Oops.
Re: LevelDB: A Fast Persistent Key-Value Store
#54Earlier quoted context omitted.
> LevelDB is slower with random reads, but that doesn't mean you shouldn't use it for unordered data - it's still quite fast. In a positive analysis (should rather than shouldn't), assuming no default choice, it seems rational to use Tokyo Cabinet or CDB _hashmaps_ for unordered data, and LevelDB for ordered data, from a datastructure and performance standpoint. To assert more would probably need a specific use case…
In a positive analysis (should rather than shouldn't), assuming no default choice, it seems rational to use Tokyo Cabinet or CDB _hashmaps_ for unordered data, and LevelDB for ordered data, from a datastructure and performance standpoint. It's as rational as doing optimizations. If the specific performance is extremely critical, yes, it definitely makes sense. But LevelDB does well with random reads [1]. With all its…
> and its permissive license
Other variables such as licensing are context dependent. In my context, I generally use embedded databases in shared library form to store arbitrary serialized Lua (much faster to reload than JSON, no need to compile protocol buffers) on my own servers, so in my development context LGPL vs BSD is irrelevant. But perhaps not for an iPhone developer. TokyoCabinet already has excellent bindings for nearly every language I've used, but thats irrelevant to a developer whose application is also C.
> Compare today's benchmarks with these http://fallabs.com/tokyocabinet/benchmark.pdf, it looks like random reads in LevelDB are quite faster than BDB.
I'd bet BDB is a snail, BUT the benchmark you just linked tests 0 databases in common with the one released by Google. What can be observed is that using a hashmap (TC) was over 7X faster than randomly accessing the fastest ordered data structure by the same author (TC-BT-RND) :)
Re: LevelDB: A Fast Persistent Key-Value Store
#55As many of the comments are, "how does it compare to xyz?", the real question is, why the fuck do we need another key-value store?
Re: LevelDB: A Fast Persistent Key-Value Store
#56Sorry if this is a bit off topic but it seems to me like most of Google's opensource projects are more source available than open source. Do they actually take contributions from the community or are they all like android, source made available when its "ready for public consumption"?
LevelDB sounds like something I would like to contribute to but if the reception is going to be chilly I won't bother, maybe pick up mongo or redis instead.
Re: LevelDB: A Fast Persistent Key-Value Store
#57An interesting development a while back that I'm surprised hasn't received more attention was Oracle's release of a SQLite-based interface to BDB: http://www.oracle.com/technetwork/database/berkeleydb/overvi... It's essentially drop-in compatible with SQLite, but with added concurrency and speed for most operations. (The concurrency addresses a major issue usually keeping SQLite as a prototyping/single-user-only opti…
> SQLite-based interface to BDB
One thing I didn't get about SQL API for BDB, how does something like
select * from users where name!='tom'
Re: LevelDB: A Fast Persistent Key-Value Store
#58Hi there! I'm a YC alum (reMail W09) and helped Jeff and Sanjay with LevelDB. Let me know if you have any questions about LevelDB and I'll see if I can help.
How does this compare to other persistent key-value stores such as Membase?
Membase is a clustered data storage service your application uses.
LevelDB is a persistence library.
That makes LevelDB the kind of thing you plug into membase to get the unique properties it has to offer (or at least for fun).
Re: LevelDB: A Fast Persistent Key-Value Store
#59Anyone know how LevelDB compares to Voldemort? From a cursory glance, they are identical in their simple API (get, put, delete)
Voldemort developer here--
Voldemort to LevelDB is what MySQL is to InnoDB: Voldemort is a distributed system that allows multiple engines to be plugged in. Mostly commonly, companies use either BerkeleyDB or MySQL as a storage engine. LinkedIn, Mendeley, EBay and others also use the read only storage engine, where the data is pre-built in Hadoop and loaded into Voldemort.
I am really excited about LevelDB: while there are higher priority projects on my plate right now, we'd very much like to see a LevelDB storage engine. If anyone is interested in contributing one, they're welcome.
The steps are:
1) Creating JNI bindings to LevelDB (or creating a .so version of LevelDB and creating JNA bindings)
2) Implementing the StorageEngine interface with the bindings, including passing in any configuration.
Here is an example of a third party InnoDB/Haildb storage engine for Voldemort:
Re: LevelDB: A Fast Persistent Key-Value Store
#60LSM-Tree is good!