Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We put a distributed database in the browser and made a game of it

tigerbeetle.com

Post reply on HN
Print view View original

41–50 of 64 posts

Re: We put a distributed database in the browser and made a game of it

#41

Post by jazzyjackson »

eatonphil wrote: post #6
brimstedt wrote: post #4

Is this something similar to couch/pouchdb?

TigerBeetle is more domain-specific, i.e. focused on financial transactions and high-performance, high-availability. There are just two entity types in the database: accounts and transfers between accounts. More details here https://docs.tigerbeetle.com/design/data-modeling if you're interested!

In comparison to {C,P}ouchDB, I think the question is around offline-first availability. Can a mobile client interact with a local database without a degraded experience and expect changes to be synchronized once a connection is re-established. I would suppose in the financial transaction space this is not A Thing.

Re: We put a distributed database in the browser and made a game of it

#42

Post by jlokier »

eatonphil wrote: post #6
brimstedt wrote: post #4

Is this something similar to couch/pouchdb?

TigerBeetle is more domain-specific, i.e. focused on financial transactions and high-performance, high-availability. There are just two entity types in the database: accounts and transfers between accounts. More details here https://docs.tigerbeetle.com/design/data-modeling if you're interested!

That sounds similar to the account state storage used in blockchains. In some of those, high performance and high space consumption are technical challenges, and the tendancy to have a lot of random keys (hashes) adds another.

I wonder if TigerBeetle would be suited to those storage performance challenges, and conversely if the low-level storage optimisations in certain implementations of blockchain account history would be more broadly applicable to the financial applications targeted by TigerBeetle.

Re: We put a distributed database in the browser and made a game of it

#43

Post by eatonphil »

jazzyjackson wrote: post #41
eatonphil wrote: post #6

Earlier quoted context omitted.

TigerBeetle is more domain-specific, i.e. focused on financial transactions and high-performance, high-availability. There are just two entity types in the database: accounts and transfers between accounts. More details here https://docs.tigerbeetle.com/design/data-modeling if you're interested!

In comparison to {C,P}ouchDB, I think the question is around offline-first availability. Can a mobile client interact with a local database without a degraded experience and expect changes to be synchronized once a connection is re-established. I would suppose in the financial transaction space this is not A Thing.

> In comparison to {C,P}ouchDB, I think the question is around offline-first availability.

Got it, thanks! Yeah that is indeed not how TigerBeetle works. If you ever cannot connect to the cluster, you keep retry messages (idempotently) until you connect and the message succeeds.

Re: We put a distributed database in the browser and made a game of it

#44

Post by eatonphil »

jlokier wrote: post #42
eatonphil wrote: post #6

Earlier quoted context omitted.

TigerBeetle is more domain-specific, i.e. focused on financial transactions and high-performance, high-availability. There are just two entity types in the database: accounts and transfers between accounts. More details here https://docs.tigerbeetle.com/design/data-modeling if you're interested!

That sounds similar to the account state storage used in blockchains. In some of those, high performance and high space consumption are technical challenges, and the tendancy to have a lot of random keys (hashes) adds another. I wonder if TigerBeetle would be suited to those storage performance challenges, and conversely if the low-level storage optimisations in certain implementations of blockchain account history w…

Yes it is similar, but simpler! We do chat with a few companies using blockchain, looking at TigerBeetle for better throughput.

> if the low-level storage optimisations in certain implementations of blockchain account history would be more broadly applicable to the financial applications targeted by TigerBeetle.

Maybe! Any examples you're thinking of?

Re: We put a distributed database in the browser and made a game of it

#45

Post by schmichael »

jorangreef wrote: post #39
schmichael wrote: post #31

This is incredible! I am curious about this paragraph though: > You’re going to see view changes when the primary crashes or is partitioned, and VSR’s telltale round robin rotation of the new primary among replicas, until a new primary is established. This is in contrast to Raft, which elects a primary at random, but then suffers from the risk of (or increased latency to mitigate) dueling leaders. It seems like regar…

Thanks, great to hear you enjoyed it! The round robin "view change" in VSR is still consensus, and uses quorums to do fault isolation of the old primary, and to preserve the intersection property, to ensure that the committed log survives into the new view. What's cool about VSR's consensus though, is that the dice is also preloaded, ahead of time, so that there's more information baked into the protocol than with Ra…

Thanks for the details. A coworker pointed me at Heidi Howard (of Flexible Paxos, etc) and Diego Ongaro (Raft) discussing this very question! https://groups.google.com/g/raft-dev/c/cBNLTZT2q8o?pli=1

Re: We put a distributed database in the browser and made a game of it

#46

Post by imtringued »

chubot wrote: post #23
petrosagg wrote: post #15

> Sure, we’re not yet injecting storage faults, but then formal proofs for protocols like Raft and Paxos assume that disks are perfect, and depend on this for correctness? After all, you can always run your database over RAID, right? Right? > If your distributed database was designed before 2018, you probably couldn’t have done much. The research didn’t exist. I'm trying to understand this part but something seems of…

Not an expert in this area, but I think disks have correlated failure modes whereas CPUs and memory generally don't. Especially spinning platter disks, not sure about SSDs. The difference in failure rates could be orders of magnitude ... Memory will have random bit flips but I think they are pretty randomly distributed (or maybe catastrophic if there is some cosmic event) But disks will have non-random manufacturing…

SSDs tend to have highly correlated failure modes because you either run into a bug in the firmware which is the same on every SSD or you have the same wear on every SSD, which locks both into read-only mode within a short period of time. You might argue that read-only is not a failure, but read-only means downtime and replacing hardware.

Re: We put a distributed database in the browser and made a game of it

#48

Post by jlokier »

eatonphil wrote: post #44
jlokier wrote: post #42

Earlier quoted context omitted.

That sounds similar to the account state storage used in blockchains. In some of those, high performance and high space consumption are technical challenges, and the tendancy to have a lot of random keys (hashes) adds another. I wonder if TigerBeetle would be suited to those storage performance challenges, and conversely if the low-level storage optimisations in certain implementations of blockchain account history w…

Yes it is similar, but simpler! We do chat with a few companies using blockchain, looking at TigerBeetle for better throughput. > if the low-level storage optimisations in certain implementations of blockchain account history would be more broadly applicable to the financial applications targeted by TigerBeetle. Maybe! Any examples you're thinking of?

> Any examples you're thinking of?

The three I know about are all Ethereum account/state history stores.

Besu does something interesting they call Bonsai Trees. It reportedly uses a lot less storage (about 1/10, 1.1TB instead of 12TB) than what came before in certain modes that store all the account history ("archive node"), and a bit less storage in other modes. I haven't studied how Bonsai trees work, so I don't know if the method is useful for non-blockchain styles of account history.

Erigon is well known as using perhaps the smallest storage when storing all the account history, and this is what it's optimised for. Erigon uses a fairly classic B-tree, for performance reasons (lower IOPS than LSM-trees for relevant operations). Their tricks are in how the data is organised in the B-tree, being better than the classic Ethereum methods. Erigon 2/3/4 (not sure, they merged plans) uses a new storage format that is supposed to do the same thing better. I don't think Erigon's tricks would be applicable to TigerBeetle, because TB doesn't have the storage problem of keeping everythng in Merkle trees in the first place. But it's a place to look as Ethereum's leader for archive node storage.

Last but not least, I have a work in progress storage format that is more efficient than either of the above, in that it uses much less storage for all the account history (due to efficient queryable and updatable adaptive compression), and fewer IOPS to look up individual account states (because random access is important). It uses a novel storage tree structure that is neither a B-tree nor an LSM-tree, but has some aspects of both, in order to reach for lower IOPS in various scenarios that each of those trees ar better or worse at, and it is also compresses the data in a novel way, which is important to storing histories and patterned data efficiently. It was originally motivated by Ethereum state history storage (since there's a clear benchmark), but I'm thinking it could be useful for time series data.

Re: We put a distributed database in the browser and made a game of it

#49

Post by vidarh »

planckscnst wrote: post #40
chubot wrote: post #23

Earlier quoted context omitted.

Not an expert in this area, but I think disks have correlated failure modes whereas CPUs and memory generally don't. Especially spinning platter disks, not sure about SSDs. The difference in failure rates could be orders of magnitude ... Memory will have random bit flips but I think they are pretty randomly distributed (or maybe catastrophic if there is some cosmic event) But disks will have non-random manufacturing…

I work on a large distributed database system. SSDs absolutely have correlated failures. Also CMOS batteries. Also CPU and memory (think a manufacturing defect or a storage climate issue on specific batches that made it through QA). Pretty much nothing is 100% guaranteed to have no correlated failures. It comes down to probabilities. You can add flexibility, variation, vendor/sourcing diversity to reduce risks.

Went through a rather large batch of OCZ SSDs that all failed within a two week window years ago. Thankfully the IBM Death Star had long before made me allergic to putting devices of the same model in the same RAID array if I can help it, so it was a nuisance rather than a disaster.

Re: We put a distributed database in the browser and made a game of it

#50

Post by VectorLock »

captainhorst wrote: post #9
Digikid13 wrote: post #8

It's very confusing that y'all are calling this a "game". There's nothing to play, it's just a simulation to watch.

There are tools in the upper right corner. You can pick one and use it on a beetle. ;) This introduces various faults in the simulation that the database has to recover from. Minor spoiler: there's another tiny game hidden at the end of the third level.

Out of curiosity what did you use to write the graphical part of this game?
Post reply on HN