Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Graviton Database: ZFS for key-value stores

github.com

Post reply on HN
Print view View original

1–10 of 59 posts

Re: Graviton Database: ZFS for key-value stores

#2

Post by ysleepy »

Nice!

I implemented pretty much the same trade off set in an authenticated storage system.

single writer, radix merkle tree, persistent storage, hashed keys, proofs.

I guess it is a local maxima within that trade off space.

I like how the time travelling/history is always touted as a feature (which it is), but it really just means the garbage collector/pruning part of the transaction engine is missing. Postgres and other mvcc systems could all be doing this, but they don't. The hard part of the feature is being able to turn it off.

I'll probably have a look around later, the diffing looks interesting, not sure yet if it's done using the merkle tree (likely) or some commit walking algorithm.

Re: Graviton Database: ZFS for key-value stores

#6

Post by carlosf »

nickcw wrote: post #5

What I'd really like is a multiprocess safe embeddable database written in pure Go. So a database which is safe to read and write from separate processes. Unfortunately I don't think this one is multiprocess safe.

Not sure if easy to embed, but Consul does offer strong consistency if explicitly configured.

Re: Graviton Database: ZFS for key-value stores

#8

Post by jopari »

Rochus wrote: post #4

What is the use case? Why is it important that "All keys, values are backed by blake 256 bit checksum"?

It seems to be intended as the backend database for the Dero blockchain smart contract platform: https://medium.com/deroproject/graviton-zfs-for-key-value-st...

The post claims: "The features included in Graviton provide the missing functionality that prevented Stargate RC1 from reaching deployment on our mainnet."

I'm not sure, but I guess that this checksumming is relevant for storing the Merkle trees encoding the blockchain. I don't know why the previous choice of database wasn't suitable.

Re: Graviton Database: ZFS for key-value stores

#9

Post by mulander »

ysleepy wrote: post #2

Nice! I implemented pretty much the same trade off set in an authenticated storage system. single writer, radix merkle tree, persistent storage, hashed keys, proofs. I guess it is a local maxima within that trade off space. I like how the time travelling/history is always touted as a feature (which it is), but it really just means the garbage collector/pruning part of the transaction engine is missing. Postgres and o…

> I like how the time travelling/history is always touted as a feature (which it is), but it really just means the garbage collector/pruning part of the transaction engine is missing. Postgres and other mvcc systems could all be doing this, but they don't.

Postgres actually did tout it as a feature in "THE IMPLEMENTATION OF POSTGRES" by Michael Stonebraker, Lawrence A. Rowe and Michael Hirohama[1] search for "time travel" in the PDF. I added the relevant quotes below for easier access ;)

This was back when PostgreSQL had the postquel language (before SQL was added) there was special syntax to access data at specific points in time:

> The second benefit of a no-overwrite storage manager is the possibility of time travel. As noted earlier, a user can ask a historical query and POSTGRES will automatically return information from the record valid at the correct time.

Quoting the paper again:

> For example to find the salary of Sam at time T one would query:

    retrieve (EMP.salary)
    using EMP [T]
    where EMP.name = "Sam"
> POSTGRES will automatically find the version of Sam’s record valid at the correct time and get the appropriate salary.

[1] - https://dsf.berkeley.edu/papers/ERL-M90-34.pdf

Post reply on HN