Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Antidote: CRDT-based distributed database

syncfree.github.io

Post reply on HN
Print view View original

31–40 of 89 posts

Re: Antidote: CRDT-based distributed database

#31

Post by marknadal »

joaomlneto wrote: post #14
xinsight wrote: post #4

The video lists pros/cons for strongly consistent and eventually consistent databases, but only has pros for a "just right consistency" database. What are the cons?

The idea of the "just right consistency" is that it brings the best of both worlds, without any drawbacks. Your application works as well as if it was executed fully in strong consistency, but with improved scalability for the set of operations that can execute in an eventual consistent model. https://www.youtube.com/watch?v=HJjWqNDh-GA

Great video. I'm impressed you guys built the checker tool, that isn't an easy task - we built something similar called PANIC (https://github.com/gundb/panic-server) that lets us simulate failure cases and then run the test across a distributed system to see if it passes.

I had a question, in the video at a certain point you say that you must modify the code to disallow concurrent debits. This makes sense in theory, but wouldn't it fail in practice? If two machines in different regions are running this code, they would not know that there is a concurrent debit. How si that addressed?

Re: Antidote: CRDT-based distributed database

#32

Post by fiatjaf »

I wanted a database that would receive "events" asynchronously and stored that, but at the same time would process these events (from some piece of previously written code) to generate a queryable schema.

If I wanted to change the schema later, the database would let me just rewrite the code and it would reprocess all the received events since the beggining.

My use case is not anything high-performance or with thousands of writes -- it's the opposite.

Re: Antidote: CRDT-based distributed database

#33

Post by skrebbel »

nine_k wrote: post #24
skrebbel wrote: post #5

At first glance, this looks amazing. I truly believe CRDTs are the solution to lots of distributed systems problems, and that exposing their characteristics to developers directly, rather than trying to abstract them away in nicer, but leaky, abstractions, is the right way to go. That said, a major part of why databases are hard is that reliable storage is hard. I see remarkably little about this on Antidote's homepa…

It appears to use Riak as the storage layer.

Ah great! That makes this very cool all of a sudden. How did I miss that?

Re: Antidote: CRDT-based distributed database

#35

Post by davidw »

I wonder what it looks like in terms of resource usage.

I think there's a strong case for something like this for IoT type devices. Imagine the simple case of adding a name to a contact list on a mobile device, and wanting that to get synced up with a series of other devices.

Re: Antidote: CRDT-based distributed database

#36

Post by Xeoncross »

davidw wrote: post #35

I wonder what it looks like in terms of resource usage. I think there's a strong case for something like this for IoT type devices. Imagine the simple case of adding a name to a contact list on a mobile device, and wanting that to get synced up with a series of other devices.

It's written in erlang so should be better ram usage than a Java database.

Re: Antidote: CRDT-based distributed database

#37

Post by KirinDave »

nine_k wrote: post #25
KirinDave wrote: post #19

Earlier quoted context omitted.

Financial examples are bad because in fact the financial world IS eventually consistent. It's quite possible to withdraw the same $100 from an account via multiple ATM machines.

With ATMs and debit cards, I thinks it's generally not true, they seem to use the online mode and update the balance of a checking account within seconds. With credit cards, you can indeed start more transactions against the same balance, and you're never sure in which order they will complete.

Gonna repeat, it is DEFINITELY possible to spend the same money multiple times with one ATM card, using old ATM machines with other payment methods.

No, I will not further detail how here.

Re: Antidote: CRDT-based distributed database

#38

Post by cmeiklejohn »

davidw wrote: post #35

I wonder what it looks like in terms of resource usage. I think there's a strong case for something like this for IoT type devices. Imagine the simple case of adding a name to a contact list on a mobile device, and wanting that to get synced up with a series of other devices.

(committer on Antidote, author of Lasp, both SyncFree projects)

Antidote is designed for DC deployments, in the hundreds of clusters. It provides causal consistency and transactions.

Lasp is designed for high-churn, edge deployments (with CRDTs) and provides eventual consistency: it's designed for 1,000+ nodes.

They share a bit of code, and you might want to evaluate both to determine which is a better use case: Lasp might be a better fit for the IoT application, because it was designed for that.

Here's a HN post on Lasp's scalability:

https://news.ycombinator.com/item?id=15444156

Here's a HN post on Lasp:

https://news.ycombinator.com/item?id=14300763

Re: Antidote: CRDT-based distributed database

#39

Post by KirinDave »

teleclimber wrote: post #27
KirinDave wrote: post #18

Earlier quoted context omitted.

It's a government sponsored project so one would expect their publicity to be limited. That's part of the reason I posted it. Stuff like this is extremely exciting. It's at the cost of playing by the rules of CRDTs. Making CRDTs consequence-free is ongoing research.

> Making CRDTs consequence-free is ongoing research. What do you mean by that? I found possibly related language on this page the other day [0]: CRDTs are "[t]ypically not suited for editing application with consequent UI." Can you point me in the right direction? My googling got me nowhere. Thanks. [0] https://irisate.com/collaborative-editing-solutions-round-up...

I dunno about that quote, Treedoc works pretty well.

In general there are tradeoffs to CRDTs and not everyone loves those tradeoffs. As time goes on we find CRDTs that make better tradeoffs.

For example, the earliest examples of CRDT sets grew linearly with the values added to the set, without regard for deletion. For all time. That's a pretty steep cost.

Re: Antidote: CRDT-based distributed database

#40

Post by KirinDave »

erik_seaberg wrote: post #29
nine_k wrote: post #25

Earlier quoted context omitted.

With ATMs and debit cards, I thinks it's generally not true, they seem to use the online mode and update the balance of a checking account within seconds. With credit cards, you can indeed start more transactions against the same balance, and you're never sure in which order they will complete.

When a payment processing system doesn't promptly cancel auth holds, customers definitely complain about being prevented from spending their money. This stuff should be table stakes but some retail banks are just way behind.

That is the reverse problem. Flippin' gas stations STILL have a problem with this that you feel acutely because the large holds. A similar problem exists with hotels, where they'll put a lock on a ton of money in your account.
Post reply on HN