Comparison to Badger? Badger is also go-native and, for me, has been exceptional at scale and for read-heavy workloads on SSD.
Graviton Database: ZFS for key-value stores
21–30 of 59 posts
Re: Graviton Database: ZFS for key-value stores
#22What I'd really like is a multiprocess safe embeddable database written in pure Go. So a database which is safe to read and write from separate processes. Unfortunately I don't think this one is multiprocess safe.
Badger supports concurrent ACID.
Re: Graviton Database: ZFS for key-value stores
#23... doesn't cassandra do a lot of this?
Cassandra is not, traditionally/practically, an embedded db.
Re: Graviton Database: ZFS for key-value stores
#24Earlier quoted context omitted.
ZFS stores the checksums of files to prevent bit rotting. Since they are comparing their database to ZFS, I guess it stores the checksums for the same reason. If bit rotting occurs, you don't need to discard the entire database, just the affected entry. If the entry was already there for some time, you might even be able to restore it from a backup.
Isn't a 256-bit Blake hash a little OTT, versus a simple CRC, or even a faster, smaller hash like MurmurHash or Jenkins-one-at-a-time?
It's a cryptographic hash, so it will detect tampering with the data, which a simple CRC, MurmerHash or Jenkins would not.
Re: Graviton Database: ZFS for key-value stores
#25If latency and performance is an issue there are also solutions like RocksDB or LevelDB
There's a brief comparison with RocksDB and LevelDB in the README file, which concludes: "If you require a high random write throughput or you need to use spinning disks then LevelDB could be a good choice unless there are requirements of versioning, authenticated proofs or other features of Graviton database."
Re: Graviton Database: ZFS for key-value stores
#26What is the use case? Why is it important that "All keys, values are backed by blake 256 bit checksum"?
ZFS stores the checksums of files to prevent bit rotting. Since they are comparing their database to ZFS, I guess it stores the checksums for the same reason. If bit rotting occurs, you don't need to discard the entire database, just the affected entry. If the entry was already there for some time, you might even be able to restore it from a backup.
I can understand it with a file system; but in a typical key/value store application the data elements are much smaller (likely even smaller than the hash result).
Re: Graviton Database: ZFS for key-value stores
#27Comparison to Badger? Badger is also go-native and, for me, has been exceptional at scale and for read-heavy workloads on SSD. Ref: https://github.com/dgraph-io/badger
I think the key differentiating feature of Graviton is the tree of authenticated proofs of data consistency. (AFAICT this is particularly important for scalably updating and verifying a large blockchain history.)
Re: Graviton Database: ZFS for key-value stores
#28I love the idea but I think you (author) need a lot of time/support polishing this. You need a team probably. Also, >Superfast proof generation time of around 1000 proofs per second per core. Does this limit in any way things like read/write perfomance or usability in general?
I asked the devs, and apparently "proof generation won't affect general read/write performance". But I guess this is the kind of thing where benchmarks might be useful, and there don't seem to be any public yet.
Re: Graviton Database: ZFS for key-value stores
#29Earlier quoted context omitted.
Isn't a 256-bit Blake hash a little OTT, versus a simple CRC, or even a faster, smaller hash like MurmurHash or Jenkins-one-at-a-time?
It's a cryptographic hash, so it will detect tampering with the data, which a simple CRC, MurmerHash or Jenkins would not.
Still, I'd like an option to use a faster, more efficient CRC or hash - bit rot is usually the main threat, rather than tampering. Not to mention that if a user can tamper with the data they can probably just create a new hash at the same time.
Using a cryptographic hash as a souped-up CRC seems rather odd, given how many more CPU cycles and RAM it will use, but I don't know the reasoning behind the decision; there must be one.
Re: Graviton Database: ZFS for key-value stores
#30Earlier quoted context omitted.
It's a cryptographic hash, so it will detect tampering with the data, which a simple CRC, MurmerHash or Jenkins would not.
Still, I'd like an option to use a faster, more efficient CRC or hash - bit rot is usually the main threat, rather than tampering. Not to mention that if a user can tamper with the data they can probably just create a new hash at the same time. Using a cryptographic hash as a souped-up CRC seems rather odd, given how many more CPU cycles and RAM it will use, but I don't know the reasoning behind the decision; there m…
I meant to say "attacker", rather than "user".