Live data from Hacker News

Data consistency is overrated

two-wrongs.com

41–50 of 72 posts

Re: Data consistency is overrated

#42
I'm a bit puzzled by the article.

First off, I'm not sure whether they consider that eventual consistency is not consistency.

Secondly, they quote a long example where business behaviour inherently results in inconsistent data. There is a significant difference between your system being robust enough to let inconsistent data exist, and to give the tools for business users to reconcile it, and a system that is not consistent iself, and lets the users deal with it.

An illustration of this is text editing. If you simply share a text file with multiple people and let everyone edit, it'll be hard to avoid people overriding each other in a chaotic and destructive way. But if you use e.g. CRDTs, you give people the ability to each follow their train of thoughts, and to reconcile their versions later on. That example tends to show how valuable consistency is, and how important it is to build a system that lets users reconcile data if the business logic is not always consistent.

Re: Data consistency is overrated

#43

Durability is also overrated. Jokes aside, I’ve been responsible for a system that processed ~1 billion monetary transactions per day. Even with a fanatical focus on consistency and correctness it was still always off. With the philosophy promoted on the OP the chaos would have been complete… is my gut feeling at least.

Durability often is overrated.

E.g. lets say your system is storing billing records for an API. Customers won't complain about being under-charged, so a loss of data for a system like that is fine as long as reducing the durability saves more money than it costs in losts ability to charge your customers. E.g. if rolling up and syncing summaries to disk every 10 seconds were to on average lose you 0.1% of records but cut your server cost for that system by 80%, it's a simple question of whether those server costs are higher or lower than .1% of your revenue (all numbers taken out of thin air).

We make sacrifices in terms of both consistency and durability and all kinds of things all the time. The problem is that we often don't take the time to analyse what kind of inconsistency and loss is acceptable from a business point of view before designing a solution that often tries to aim for a level of consistency or durability that may not be needed.

You can afford to sacrifice all kinds of guarantees as long as you have reliable models for how sacrificing them impacts you and your customers so that you know the sacrifices represent acceptable tradeoffs. The risk you're taking is that you're sacrificing safety margins, and so it's something you should only do if you understand both the domain and your systems failure modes very well.

Re: Data consistency is overrated

#44
post #38

The problem with no consistency isn't the impact to users, but developers. If things can be inconsistent you need to worry about a whole new class of failure cases where data is inconsistent, and handle them correctly. Part of this is guarantees about correctness. Is there a bug that manifests even without inconsistency, or was there just some weird inconsistent state that caused an issue? If you don't have consisten…

One hundred percent agree

Re: Data consistency is overrated

#45
I find the example given by the author about financial inconsistency to be alarming. When you consider the fact that the monetary system is made up of large numbers of participants and all of them are constantly making small mistakes... Surely it can accumulate into significant discrepancies over time.

Imagine that you create a multiplayer MMORPG video game with a 'limited' number of virtual gold coins and you set the supply at 1 billion gold coins and you decide to track account balances using 1000 different systems managed by different companies (which constantly credit and debit each other using different mechanisms) instead of just 1 system managed by 1 company... What are the odds that you will still have only 1 billion gold coins in circulation in one year's time? Surely at least 1 of the 1000 systems will go rogue and start inflating the currency supply; it may start sending fake coins to other systems and the other systems won't be able to know that they are fake since they all operate on mutual trust instead of cryptographic proofs, they will unwittingly propagate records of those fake coins to other systems as if they were real. This is essentially how the fiat monetary system works.

The power of money creation is not supposed to be in the hands of private institutions, yet because currency is tracked across so many different systems and there is no universal consistency between them, money creation seems to be essentially guaranteed to occur; especially when you consider that the incentive is to turn a blind eye to errors which benefit large institutions.

Re: Data consistency is overrated

#46
post #7

I think there are two different kinds of consistency, and it's important to not conflate them. There's consistency that's internal to a system. Do all of the foreign keys line up correctly? Have I lost any data that was provided to me? Here, we can aspire to be 100% correct. I don't think the examples in this article conflict with that. Then there's consistency that's external to a system. This can be between this sy…

There's a great book "Data and Reality" that discusses these subtle but very crucial differences. Discussed here in HN a year ago: https://news.ycombinator.com/item?id=30251747

Thank you so much for reminding me! I remember Hillel Wayne’s post about it from a while ago, and I had only gotten through the first couple of chapters before stuff got in the way. It might be a good long weekend read.

Great suggestion!

Re: Data consistency is overrated

#47

I once worked on a platform producing analytics using data that, at its source, was manually typed in by people. My product managers would insist we do distinct counts on the aggregates instead of using probabilistic algorithms, because we "needed" the absolute 100% accurate output. No matter how many times I would explain the data was never 100% accurate to begin with and that the error rate using HyperLogLog wouldn…

Why would your PM dictate the technical implementation? This seems like something better left to engineers.

Re: Data consistency is overrated

#48
post #7

I think there are two different kinds of consistency, and it's important to not conflate them. There's consistency that's internal to a system. Do all of the foreign keys line up correctly? Have I lost any data that was provided to me? Here, we can aspire to be 100% correct. I don't think the examples in this article conflict with that. Then there's consistency that's external to a system. This can be between this sy…

Sort of reminds me of memory consistency and gpus.

In the beginning, there was less need for it - suuuper fast memory, but you might have a bit error on a pixel glitch for one frame and ... no big deal.

Then GPUs started to be used for compute and... well, we might not only need good memory, we might need ECC...

Re: Data consistency is overrated

#49
Data consistency is an aspiration which we achieve by many means, including pragmatic tolerance for inconsistency, when consistency is not possible and alternate mechanisms for handling it. Those mechanisms are almost always less efficient than consistency.

So to say consistency is "overrated" is nonsense. It's not always 100% applicable, but this comes at a cost.

Re: Data consistency is overrated

#50

I once worked on a platform producing analytics using data that, at its source, was manually typed in by people. My product managers would insist we do distinct counts on the aggregates instead of using probabilistic algorithms, because we "needed" the absolute 100% accurate output. No matter how many times I would explain the data was never 100% accurate to begin with and that the error rate using HyperLogLog wouldn…

I could see myself being like your PM in some situations, depending on the nature of the data and how it was being used. To give an analogy that might perhaps help with understanding the sentiment, imagine how your fellow citizens would feel if the government used some {hot approximation algorithm} for counting votes in the next election, rather than a straightforward count.
Post reply on HN