Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RocksDB – A persistent key-value store for fast storage environments

rocksdb.org

Post reply on HN
Print view View original

21–30 of 75 posts

Re: RocksDB – A persistent key-value store for fast storage environments

#21

Post by alisson »

AaronFriel wrote: post #5
rdtsc wrote: post #3

Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…

The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance? Something seems very wrong with these benchmarks: Section 5 (SSD) F (Synchronous Writes) Random Writes LevelDB 342 ops/sec Kyoto TreeDB 67 ops/sec SQLite3 114 ops/sec MDB 148 ops/sec MDB, no MetaSync 322 ops/sec BerkeleyDB 291 ops/sec Section 8 (HDD) F (Synchronous Writes) Random Writes LevelDB 1291 o…

I'm not sure but I think HDD are better at sequential operations than SSD (which performs better at random operations). Some people say MySQL performs better on a high speed HDD than on a SSD.

Re: RocksDB – A persistent key-value store for fast storage environments

#22

Post by snewman »

Very nice work, and the wiki is also quite nice -- I wish more projects had a page like https://github.com/facebook/rocksdb/wiki/Rocksdb-Architectur.... It's really nice to see a clear, terse summary of what makes this project interesting relative to its predecessors.

At my company (scalyr.com), we've built a more-or-less clone of LevelDB in Java, with a similar goal of extracting more performance on high-powered servers (and better integration with our Java codebase). I'll be digging through rocksdb to see what ideas we might borrow. A few things we've implemented that might be interesting for rocksdb:

* The application can force segments to be split at specified keys. This is very helpful if you write a block of data all at once and then don't touch it for a long time. The initial memtable compaction places this data in its own segment and then we can push that segment down to the deepest level without ever compacting it again. It can also eliminate the need for bloom filters for many use cases, as you often wind up with only one segment overlapping a particular key range.

* The application can specify different compression schemes for different parts of the keyspace. This is useful if you are storing different kinds of data in the same database.

* We don't use timestamps anywhere other than the memtable. This puts some constraints on snapshot management, but streamlines get/scan operations and reduces file size for small values.

Do you have benchmarks for scan performance? This is an important area for us. I don't have exact figures handy, but we get something like 2GB/second (using 8 threads) on an EC2 h1.4xlarge, uncached (reading from SSD) and decompressing on the fly. This is an area we've focused on.

I'd enjoy getting together to compare notes -- send me an e-mail if you're interested. steve @ (the domain mentioned above).

Re: RocksDB – A persistent key-value store for fast storage environments

#23

Post by hyc_symas »

justin66 wrote: post #14
AaronFriel wrote: post #5

Earlier quoted context omitted.

The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance? Something seems very wrong with these benchmarks: Section 5 (SSD) F (Synchronous Writes) Random Writes LevelDB 342 ops/sec Kyoto TreeDB 67 ops/sec SQLite3 114 ops/sec MDB 148 ops/sec MDB, no MetaSync 322 ops/sec BerkeleyDB 291 ops/sec Section 8 (HDD) F (Synchronous Writes) Random Writes LevelDB 1291 o…

Paging hyc_symas. Howard Chu isn't shy about talking about benchmark results and he can be found here and on twitter.

Hello hello! Pretty sure what actually happened in these is that the HDD's internal cache was still active, while the Crucial M4 SSD has no internal cache. The only other explanation is that I screwed up my partition offset on the SSD but I already double-checked that and the partitions were all 2MB aligned.

Re: RocksDB – A persistent key-value store for fast storage environments

#24

Post by hyc_symas »

AaronFriel wrote: post #5
rdtsc wrote: post #3

Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…

The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance? Something seems very wrong with these benchmarks: Section 5 (SSD) F (Synchronous Writes) Random Writes LevelDB 342 ops/sec Kyoto TreeDB 67 ops/sec SQLite3 114 ops/sec MDB 148 ops/sec MDB, no MetaSync 322 ops/sec BerkeleyDB 291 ops/sec Section 8 (HDD) F (Synchronous Writes) Random Writes LevelDB 1291 o…

Keep in mind, the HDD was using ext2 and the SSD was using reiserfs. Synchronous writes on ext2 are faster than all journaling filesystems.

Re: RocksDB – A persistent key-value store for fast storage environments

#25

Post by hyc_symas »

gfodor wrote: post #2

this is cool, though I'd wonder how it compares to Kyoto Cabinet. another big issue I've run into personally is the fact that both LevelDB and KC don't explicitly support multiple processes reading the db at once. (KC's API allows this but advises against it, LevelDB afaik doesn't even allow it.) I wonder if RocksDB gets past this.

Kyoto Cabinet will self-corrupt if you use it that way. LMDB supports multi-process explicitly.

Re: RocksDB – A persistent key-value store for fast storage environments

#26

Post by hyc_symas »

snewman wrote: post #22

Very nice work, and the wiki is also quite nice -- I wish more projects had a page like https://github.com/facebook/rocksdb/wiki/Rocksdb-Architectur... . It's really nice to see a clear, terse summary of what makes this project interesting relative to its predecessors. At my company (scalyr.com), we've built a more-or-less clone of LevelDB in Java, with a similar goal of extracting more performance on high-powered se…

SkyDB using LMDB gets 3GB/sec on a standalone PC. https://groups.google.com/forum/#!msg/skydb/CMKQSLf2WAw/zBO1...

Re: RocksDB – A persistent key-value store for fast storage environments

#27

Post by hyc_symas »

rdtsc wrote: post #3

Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…

LSMs have a long long way to go to catch up to LMDB. http://symas.com/mdb/hyperdex/

Re: RocksDB – A persistent key-value store for fast storage environments

#28

Post by rdtsc »

hyc_symas wrote: post #27
rdtsc wrote: post #3

Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…

LSMs have a long long way to go to catch up to LMDB. http://symas.com/mdb/hyperdex/

Howard, is LMDB effectively limited to 128T (on 64bit machines and 2GB on 32bit ones, not that one should be running large databases on 32bit machines)?

Also what about concurrent writes? Does it have a database wide writer lock or is it per key (per page?) ?

Re: RocksDB – A persistent key-value store for fast storage environments

#29

Post by hyc_symas »

rdtsc wrote: post #28
hyc_symas wrote: post #27

Earlier quoted context omitted.

LSMs have a long long way to go to catch up to LMDB. http://symas.com/mdb/hyperdex/

Howard, is LMDB effectively limited to 128T (on 64bit machines and 2GB on 32bit ones, not that one should be running large databases on 32bit machines)? Also what about concurrent writes? Does it have a database wide writer lock or is it per key (per page?) ?

It is limited to the logical address space. Since most current x86-64 machines have only 48bit address space, 256TB, and assuming the kernel keeps half of the space for itself, then yes, the current limit is 128TB. But I suspect we'll be seeing 56bit address spaces fairly soon.

It is a single-writer DB, one DB-wide writer lock. Fine-grained locking is a tar pit.

Re: RocksDB – A persistent key-value store for fast storage environments

#30

Post by rdtsc »

hyc_symas wrote: post #29
rdtsc wrote: post #28

Earlier quoted context omitted.

Howard, is LMDB effectively limited to 128T (on 64bit machines and 2GB on 32bit ones, not that one should be running large databases on 32bit machines)? Also what about concurrent writes? Does it have a database wide writer lock or is it per key (per page?) ?

It is limited to the logical address space. Since most current x86-64 machines have only 48bit address space, 256TB, and assuming the kernel keeps half of the space for itself, then yes, the current limit is 128TB. But I suspect we'll be seeing 56bit address spaces fairly soon. It is a single-writer DB, one DB-wide writer lock. Fine-grained locking is a tar pit.

Makes sense.

Most impressive about LMDB to me is the zero-copy model for readers, with is no extra memcpy needed, maybe that is something obvious for database gurus but it is pretty clever trick I think.

Post reply on HN