Tnx for all the comments! Feel free to continue the discussion at https://www.facebook.com/groups/rocksdb.dev/
RocksDB – A persistent key-value store for fast storage environments
71–75 of 75 posts
Re: RocksDB – A persistent key-value store for fast storage environments
#72Very nice work, and the wiki is also quite nice -- I wish more projects had a page like https://github.com/facebook/rocksdb/wiki/Rocksdb-Architectur... . It's really nice to see a clear, terse summary of what makes this project interesting relative to its predecessors. At my company (scalyr.com), we've built a more-or-less clone of LevelDB in Java, with a similar goal of extracting more performance on high-powered se…
Thanks for your comments Steve.
1. RocksDb has a feature that allows an application to determine when to close a file (i.e. segment). You can write your compaction code via compaction_filter_factory defined in https://github.com/facebook/rocksdb/blob/master/include/rock...
2. RocksDb also has a feature that allows an application to close a block inside a segment. https://github.com/facebook/rocksdb/commit/fd075d6edd68ddbc1...
3. RocksDb has a feature to use different compression algorithms for different parts of the database. In the Level Style Compaction, you can configure a different compression algorithm for different levels. In Universal Style Compaction, you can specify that you want compression only for x% earliest data in your database.
4. We have internal benchmarks for scan performance but because of lack of developer resources, we might not be able to open source those numbers.
It will be great to catch up in person.
Re: RocksDB – A persistent key-value store for fast storage environments
#73The benchmark at https://github.com/facebook/rocksdb/wiki/Performance-Benchma... states that for LevelDB, "in 24 hours it inserted only 2 million key-values", and that "each key is of size 10 bytes, each value is of size 800 bytes". I might be missing something, but that took just a few minutes on my ~2 year old desktop machine. Sample code: https://gist.github.com/wbolster/7487225
There was a typo, the 2 million should have been 200 million keys. I fixed the wiki page. Thanks again for pointing it out.
Re: RocksDB – A persistent key-value store for fast storage environments
#74Earlier quoted context omitted.
fixed your link: https://github.com/facebook/rocksdb/blob/master/db/db_impl.c... Take another look! There's a guard object used at the function scope to ensure the lock is released, and this block is bracketed to release and reacquire the lock, not acquire and release. There may be a case for a guard object that does the release/reacquire, but its definitely not a slam dunk like acquire/release
Still, that's not exception-safe, correct? If LogFlush or SleepForMicroseconds throws an exception the mutex will be unlocked twice, which pthreads disallows for normal mutexes...
[deleted]
Re: RocksDB – A persistent key-value store for fast storage environments
#75Earlier quoted context omitted.
fixed your link: https://github.com/facebook/rocksdb/blob/master/db/db_impl.c... Take another look! There's a guard object used at the function scope to ensure the lock is released, and this block is bracketed to release and reacquire the lock, not acquire and release. There may be a case for a guard object that does the release/reacquire, but its definitely not a slam dunk like acquire/release
Still, that's not exception-safe, correct? If LogFlush or SleepForMicroseconds throws an exception the mutex will be unlocked twice, which pthreads disallows for normal mutexes...
You know, for a second I thought you were wrong, but I changed my mind. This does look like a bug, and a simple on to avoid at that.
It's tough, because Rocks is still highly based on LevelDB, which conforms to Google's coding style guideline, which makes RAII more than a bit tricky to do right.