Probably a naive question, but how does it compare to Redis? When would someone look for a K-V store written in X instead of the already mature Redis?
redis is not an embedded database, while rocksdb/boltdb/badger are.
Badger – A fast key-value store written natively in Go
31–40 of 101 posts
Re: Badger – A fast key-value store written natively in Go
#32Probably a naive question, but how does it compare to Redis? When would someone look for a K-V store written in X instead of the already mature Redis?
Yes this is more comparable to level, rocks or at a higher level Cassandra, ScyllaDB, DynamoDB etc.
Re: Badger – A fast key-value store written natively in Go
#33Probably a naive question, but how does it compare to Redis? When would someone look for a K-V store written in X instead of the already mature Redis?
redis is not an embedded database, while rocksdb/boltdb/badger are.
Can you say what exactly is an "embedded workload"? I have seen this a few times now and tried googling but only ever come up with references to embedded systems and I'm guessing that this is not the same context.
I know rocksdb is LSM-based and was built by FB to address write amplification on SSDs.
Re: Badger – A fast key-value store written natively in Go
#34Earlier quoted context omitted.
redis is not an embedded database, while rocksdb/boltdb/badger are.
Can you say what exactly is an "embedded workload"? I have seen this a few times now and tried googling but only ever come up with references to embedded systems and I'm guessing that this is not the same context. I know rocksdb is LSM-based and was built by FB to address write amplification on SSDs.
It has no server that runs independently and applications connect to, instead you integrate (embed) it into your program as a library.
Re: Badger – A fast key-value store written natively in Go
#35Now let somebody run a QuickCheck on it, like they did with LevelDB: http://htmlpreview.github.io/?https://raw.github.com/strange...
Re: Badger – A fast key-value store written natively in Go
#36Figures per minute? Same-process key-value should give millions per second operations (using one thread). And accessing via network, e.g. a la Redis, should be hundreds of thousands per second [1].
Re: Badger – A fast key-value store written natively in Go
#37Probably a naive question, but how does it compare to Redis? When would someone look for a K-V store written in X instead of the already mature Redis?
Badger is designed to store data on disk (like RocksDB or LevelDB), while Redis is an in-memory storage which can't store data sets larger than memory.
You can store Redis on disk too. Not that you gain anything from doing so as persistence can be better achieved by clustering and I've never ran into issues where memory was a limiting factor, even with millions of records in Redis. Frankly if memory was a limiting factor then you'd probably want your KV store separate from your application anyway rather than the embedded approach that Badger takes.
I think the real advantage of Badger is that it's not as sophisticated as Redis. ie you can have Redis-like functionality compiled into your application so less faffing about setting up another daemon / cloud micro-service inside your NAT / VPC / whatever.
Re: Badger – A fast key-value store written natively in Go
#38This is pretty exciting. I would love to see comparisons between badger and goleveldb
Re: Badger – A fast key-value store written natively in Go
#39Silly question - if SSDs or even motherboard had persistent storage of (couple of) 4 KB blocks where you'd be able to fsync It seems that databases often want to persist/flush data in unfinished (4KB or 8KB) pages when they're being built, once they are full, they don't change much - once full they could be normally persisted. Another kind of pages are those that change very frequently - ie. single "root" page which keeps counters or other stuff in single, root page.
It seems a bit wasteful that multiple "checkpoints" (flushes/fsyncs) on partial blocks are triggering on hardware whole block rewrites. Similarly with "root"/"meta" pages that keep track of just few bytes frequently changing are triggering similar whole page rewrites.
To be honest even some kind of PCI card with little battery and slot for DDR4 would probably do the trick, no? The rest could be implemented in software - as long as you'd have access to fast flushing with battery backed memory that survives hard crash - it should be fine.
Is this silly idea?
Re: Badger – A fast key-value store written natively in Go
#40There is so much negativity in these comments! This project is really cool! I really appreciate the trend to rewrite C/C++ libraries in Go. It has always been really frustrating that hacky library wrappers for other languages leave performance and features on the table because they're either incomplete or just too hard to implement in the host language. For most of the languages out there, it is always better to have…
Was with you until you said you don't need SQL.