RocksDB – A persistent key-value store for fast storage environments
1–10 of 75 posts
Re: RocksDB – A persistent key-value store for fast storage environments
#2this is cool, though I'd wonder how it compares to Kyoto Cabinet. another big issue I've run into personally is the fact that both LevelDB and KC don't explicitly support multiple processes reading the db at once. (KC's API allows this but advises against it, LevelDB afaik doesn't even allow it.) I wonder if RocksDB gets past this.
Re: RocksDB – A persistent key-value store for fast storage environments
#3Well LevelDB is already good. And if this improves on it, that's great.
I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking:
http://hyperdex.org/performance/leveldb/
So now I am curios how it would compare.
Another interesting case optimized for reads is LMDB. That is a small but very fast embedded database at sits at the core of OpenLDAP. That one has impressive benchmarks.
http://symas.com/mdb/microbench/
(Note: LMDB used to be called MDB, you might know it by that name).
Re: RocksDB – A persistent key-value store for fast storage environments
#4this is cool, though I'd wonder how it compares to Kyoto Cabinet. another big issue I've run into personally is the fact that both LevelDB and KC don't explicitly support multiple processes reading the db at once. (KC's API allows this but advises against it, LevelDB afaik doesn't even allow it.) I wonder if RocksDB gets past this.
HyperDex?
Re: RocksDB – A persistent key-value store for fast storage environments
#5Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…
The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance? Something seems very wrong with these benchmarks:
Section 5 (SSD) F (Synchronous Writes)
Random Writes
LevelDB 342 ops/sec
Kyoto TreeDB 67 ops/sec
SQLite3 114 ops/sec
MDB 148 ops/sec
MDB, no MetaSync 322 ops/sec
BerkeleyDB 291 ops/sec
Section 8 (HDD) F (Synchronous Writes)
Random Writes
LevelDB 1291 ops/sec
Kyoto TreeDB 28 ops/sec
SQLite3 112 ops/sec
MDB 297 ops/sec
BerkeleyDB 704 ops/sec
This smells.
Re: RocksDB – A persistent key-value store for fast storage environments
#6Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…
The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance? Something seems very wrong with these benchmarks: Section 5 (SSD) F (Synchronous Writes) Random Writes LevelDB 342 ops/sec Kyoto TreeDB 67 ops/sec SQLite3 114 ops/sec MDB 148 ops/sec MDB, no MetaSync 322 ops/sec BerkeleyDB 291 ops/sec Section 8 (HDD) F (Synchronous Writes) Random Writes LevelDB 1291 o…
> The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance?
Could it be that most database engines are based on algorithms that were developed before SSDs were significant, and were extremely optimized for HDD performance?
Re: RocksDB – A persistent key-value store for fast storage environments
#7this is cool, though I'd wonder how it compares to Kyoto Cabinet. another big issue I've run into personally is the fact that both LevelDB and KC don't explicitly support multiple processes reading the db at once. (KC's API allows this but advises against it, LevelDB afaik doesn't even allow it.) I wonder if RocksDB gets past this.
If you just need concurrent reading and update the database from a single process, you can use CDB. It always served me well.
Re: RocksDB – A persistent key-value store for fast storage environments
#8Well LevelDB is already good. And if this improves on it, that's great. I was looking at embedded key value stores and also found -- HyperLevelDB (from creators of Hyperdex database). They also improved on LevelDB in respect to compaction and locking: http://hyperdex.org/performance/leveldb/ So now I am curios how it would compare. Another interesting case optimized for reads is LMDB. That is a small but very fast em…
The LMDB statistics are very strange - why is synchronous SSD performance worse on most figures than HDD performance? Something seems very wrong with these benchmarks: Section 5 (SSD) F (Synchronous Writes) Random Writes LevelDB 342 ops/sec Kyoto TreeDB 67 ops/sec SQLite3 114 ops/sec MDB 148 ops/sec MDB, no MetaSync 322 ops/sec BerkeleyDB 291 ops/sec Section 8 (HDD) F (Synchronous Writes) Random Writes LevelDB 1291 o…
I would guess the answer is SSD Write Amplification. SSDs in order to write have to erase first. They also try to minimize wear so internally they spread the data around as it gets written. Maybe someone else with more experience can explain more, but that's my guess.
Re: RocksDB – A persistent key-value store for fast storage environments
#9A very minor point:
The illustrative code snippet on the home page has a spurious semicolon on the first line:
#include ;Re: RocksDB – A persistent key-value store for fast storage environments
#10Hi guys, I am Dhruba and I work in the Database Engineering team at Facebook. We just released RocksDB as an open source project. If anybody has any technical questions about RocksDB, please feel free to ask. Thanks.