Consistency is Consistently Undervalued • Hacker News Board

Re: Consistency is Consistently Undervalued

#41

Post by [deleted] » Sat, Sep 17, 2016, 12:32 PM UTC

tscs37 wrote: post #31

brandur wrote: post #22

Amen. Whether or not the article's example is a good one, in a world without consistency you need to worry about state between _any_ two database operations in the system, so there's nearly unlimited opportunity for this class of error in almost any application found in the real world. The truly nefarious aspect of NoSQL stores is that the problems that arise from giving up ACID often aren't obvious until your new pr…

Totally agree. I've wet my toes with NoSQL systems but things like MongoDB just look like trouble waiting to happen. On the other hand, SQL makes it hard to maintain the state definition, I had to develop a mechanism to upgrade the DB from any possible state to the latest state. Still, this allows me to define very accurately what is a valid row or entry in the database, which means my Applications need to make far f…

[deleted]

Re: Consistency is Consistently Undervalued

#42

Post by qaq » Sat, Sep 17, 2016, 12:34 PM UTC

tscs37 wrote: post #31

brandur wrote: post #22

Amen. Whether or not the article's example is a good one, in a world without consistency you need to worry about state between _any_ two database operations in the system, so there's nearly unlimited opportunity for this class of error in almost any application found in the real world. The truly nefarious aspect of NoSQL stores is that the problems that arise from giving up ACID often aren't obvious until your new pr…

Totally agree. I've wet my toes with NoSQL systems but things like MongoDB just look like trouble waiting to happen. On the other hand, SQL makes it hard to maintain the state definition, I had to develop a mechanism to upgrade the DB from any possible state to the latest state. Still, this allows me to define very accurately what is a valid row or entry in the database, which means my Applications need to make far f…

"MongoDB: Because /dev/null doesn't support sharding"

Re: Consistency is Consistently Undervalued

#43

Post by sz4kerto » Sat, Sep 17, 2016, 1:03 PM UTC

The universally hated JEE can do distributed transactions by default. Yes, with pitfalls, but it can. (It is usually hated by devs who have never used it properly.)

Re: Consistency is Consistently Undervalued

#44

Post by ah- » Sat, Sep 17, 2016, 1:04 PM UTC

You can use event logs and eventual consistency to solve this problem.

Basically you make the transfer of money an event that is then atomically committed to an event log. The two bank accounts then eventually incorporate that state.

See http://www.grahamlea.com/2016/08/distributed-transactions-mi...

But I agree that often life is easier if you just keep things simpler. If you require strong consistency like with the user/profile don't make that state distributed. If you do make it distributed you need to live with less consistency.

Re: Consistency is Consistently Undervalued

#45

Post by ehwizard » Sat, Sep 17, 2016, 1:06 PM UTC

tscs37 wrote: post #31

brandur wrote: post #22

Amen. Whether or not the article's example is a good one, in a world without consistency you need to worry about state between _any_ two database operations in the system, so there's nearly unlimited opportunity for this class of error in almost any application found in the real world. The truly nefarious aspect of NoSQL stores is that the problems that arise from giving up ACID often aren't obvious until your new pr…

Totally agree. I've wet my toes with NoSQL systems but things like MongoDB just look like trouble waiting to happen. On the other hand, SQL makes it hard to maintain the state definition, I had to develop a mechanism to upgrade the DB from any possible state to the latest state. Still, this allows me to define very accurately what is a valid row or entry in the database, which means my Applications need to make far f…

Minor point, but MongoDB added document validation in 3.2 so you can add guarantees about what documents are being stored.

https://docs.mongodb.com/manual/core/document-validation/

Re: Consistency is Consistently Undervalued

#46

Post by acjohnson55 » Sat, Sep 17, 2016, 1:11 PM UTC

I sort of agree. The examples in the article are ways in which people play fast and loose with consistency, often using a NoSQL store that has poor support for atomicity and isolation. This is a helpful message, because I've definitely seen naively designed systems start to develop all sorts of corruption when used at scale. The answer for many low-throughput applications is to just use Postgres. Both Django and Rails, by default, work with relational databases and leverage transactions for consistency.

Then, there is the rise of microservices to consider. In this case, I also agree with the author that it becomes crucial to understand that the number of states your data model can be in can potentially multiply, since transactional updates are very difficult to do.

But I feel like on the opposite side of the spectrum of sophistication are people working on well-engineered eventually consistent data systems, with techniques like event sourcing, and a strong understanding of the hazards. There's a compelling argument that this more closely models the real world and unlocks scalability potential that is difficult or impossible to match with a fully consistent, ACID-compliant database.

Interestingly, in a recent project, I decided to layer in strict consistency on top of event sourcing underpinnings (Akka Persistence). My project has low write volume, but also no tolerance for the latency of a write conflict resolution strategy. That resulted in a library called Atomic Store [1].

[1] https://github.com/artsy/atomic-store

Re: Consistency is Consistently Undervalued

#47

Post by catnaroek » Sat, Sep 17, 2016, 1:32 PM UTC

socmag wrote: post #27

brandur wrote: post #22

Amen. Whether or not the article's example is a good one, in a world without consistency you need to worry about state between _any_ two database operations in the system, so there's nearly unlimited opportunity for this class of error in almost any application found in the real world. The truly nefarious aspect of NoSQL stores is that the problems that arise from giving up ACID often aren't obvious until your new pr…

A database is an abstract concept, and ACID transactions on databases have nothing to do with the interface semantics used to interact with the shared state being managed. You can have non-ACIDic SQL databases and ACID NoSQL databases. While it is true that many KV stores do not present a fully consistent view over distributed data, there are some that do. It is very important that people shed this idea that only SQL…

> While it is true that many KV stores do not present a fully consistent view over distributed data, there are some that do.

Consistency is always relative to what the DBMS can enforce, which is in turn always relative to what the the DBMS's schema language can express. A KV store can be trivially consistent in that, between writes, it always presents the same view of the data. But it won't be consistent in the sense of presenting a view of the data that's guaranteed not to contradict your business rules. The latter is what users of relational databases expect.

Re: Consistency is Consistently Undervalued

#48

Post by kahirsch » Sat, Sep 17, 2016, 1:38 PM UTC

vidarh wrote: post #25

My opinion is exactly opposite: Consistently is overvalued. Requiring consistency in distributed system generally leads to designs that reduces availability. Which is one of the reasons that bank transactions generally do not rely on transactional updates against your bank. "Low level" operations as part of settlement may us transactions, but the bank system is "designed" (more like it has grown by accretion) to func…

As you say, the low-level operations in banking do provide atomicity, which are often lacking in distributed databases.

Also, the banking system does have the system of compensating transactions (e.g. returned checks) when constraints are violated, as well as an extensive system of auditing to maintain consistency.

Do most distributed systems have the ability to detect inconsistency? How do they manage to correct problems when found? Is it mostly manual? I'm genuinely interested in hearing what people have done.

I've certainly experienced inconsistencies which were in very low-stakes online situations that were still annoying. For example, a reddit comment that was posted on a thread, but never shows up my user page. Or when I stopped watching a movie on Netflix on my PC to go watch something else with my wife on our TV, but Netflix won't let us because it thinks I'm still watching on the PC.

Come to think of it, there was also something that used to happen frequently with Amazon video, though I haven't seen it happen in a while. When I would resume a video, it gave me two choices of times to resume, one based on the device and one based on the server. The odd thing was that I had only ever used one device to watch the video but it was always the server-stored time that was right. Just a bug, I guess, but that does indicate the kind of inconsistencies that have to be resolved somehow.

Re: Consistency is Consistently Undervalued

#49

Post by bunderbunder » Sat, Sep 17, 2016, 1:48 PM UTC

vidarh wrote: post #25

My opinion is exactly opposite: Consistently is overvalued. Requiring consistency in distributed system generally leads to designs that reduces availability. Which is one of the reasons that bank transactions generally do not rely on transactional updates against your bank. "Low level" operations as part of settlement may us transactions, but the bank system is "designed" (more like it has grown by accretion) to func…

My (admittedly limited, so far) experience in the financial industry has indicated that, while it is the norm is to use settlement and reconciliation in situations where making ACID-style guarantees is impossible, ACID-style guarantees are most emphatically _not_ foregone when it isn't necessary to do so.

One big reason for that is that the settlement and reconciliation process is easier to manage when the parties involved can at least be confident that their own view of the situation is internally consistent. Gremlin-hunting is a lot less expensive if you can minimize the number of places for gremlins to hide.

Re: Consistency is Consistently Undervalued

#50

Post by socmag » Sat, Sep 17, 2016, 1:57 PM UTC

vidarh wrote: post #25

My opinion is exactly opposite: Consistently is overvalued. Requiring consistency in distributed system generally leads to designs that reduces availability. Which is one of the reasons that bank transactions generally do not rely on transactional updates against your bank. "Low level" operations as part of settlement may us transactions, but the bank system is "designed" (more like it has grown by accretion) to func…

This!

There is no single "truth".

Under the unfortunate consequences of the CAP theorem and the speed of light, consistency grinds to a halt as the number of observers increases. Time itself would have to stand still in the limit if at each time step every particle in the universe had to agree on the state of the system.

What we need to preserve is mostly strong "causal consistency", which means that from a single observers perspective, the system "appears consistent".

This article is really good and isn't obscured by NoSQL vs SQL arguments.

http://www.grahamlea.com/2016/08/distributed-transactions-mi...