Consistency is Consistently Undervalued • Hacker News Board

Re: Consistency is Consistently Undervalued

#81

Post by kpmah » Sat, Sep 17, 2016, 4:21 PM UTC

phamilton wrote: post #73

This profile example is missing the better approach: avoid the dependency of creating the user before creating the profile. Create the profile with a generated uuid. Once that succeeds, then create the user with the same uuid. If you build a system that allows orphaned profiles (by just ignoring them) then you avoid the need to deal with potentially missing profiles. This is essentially implementing MVCC. Write all y…

Well, we're playing make believe with the requirements. In fairyland the user and the profile need to exist together. Only the fairies know why!

If you can relax the requirements, you can relax the constraints.

Re: Consistency is Consistently Undervalued

#82

Post by knucklesandwich » Sat, Sep 17, 2016, 4:22 PM UTC

vidarh wrote: post #68

knucklesandwich wrote: post #60

Earlier quoted context omitted.

> You design systems to make them self-correcting: Log events durably first, and then act on the events. If possible, make operations idempotent. If not, either make operations fully commutative, or group them into larger commutative operations and use local transactions (so require local consistency) to ensure they are all applied at once. The point is not to require no atomicity, but to localise any guarantees as m…

> You can't do this in general, because we often want to build applications where the user expects their actions won't be reordered. The key being they don't expect their actions to be reordered. In most systems this is trivially handled by logging event time as seen from the users system and ensure the user sees a consistent view of their own operations . There are exceptions, naturally, but the point is not that yo…

> Your privacy settings is a local concern to your account, it most certainly does not need to be globally consistent. All you need to do to avoid problems with that is to ensure consistency in routing requests for a given session, and enforce local consistency of ordering within a given user account or session depending on requirements.

I think you're misunderstanding what the problem is here. I want global consistency (really I want two party consistency) to prevent the said user from witnessing the unkind thing I posted about them. It's necessary that my own actions appear in their original order to me, but also to anyone else who is plausibly affected by their reordering.

The full example also involves posting on a mutual friend's wall and having that post hidden by unfriending. You could route all requests to fetch posts to the owning poster's pinned set of servers, but this would be extremely expensive. Sharding solutions also weaken availability guarantees, because you necessarily shrink the set of nodes able to service a request.

Re: Consistency is Consistently Undervalued

#83

Post by bullen » Sat, Sep 17, 2016, 4:39 PM UTC

matthewmacleod wrote: post #13

bullen wrote: post #7

But consider this: You are using mysql, you make a transaction with say deposit and withdraw. What happens on the mysql machine if you pull the plug exactly when mysql has done the deposit but not the withdraw? The ONLY difference between SQL transactions and NoSQL microservice transactions is the time between the parts of a transaction. Personally I use a JSON file with state to execute my NoSQL microservice transac…

I don't mean to be unkind, but is this meant to be a parody? There are no "parts of a transaction" because a transaction is definitionally atomic. Transactional file modification is a fairly tricky problem, and I'd be surprised if you'd actually implemented a safe system in that manner. What's certain though is that spinning up MySQL or Postgres and using it to store simple records is essentially a zero-cost setup ta…

How is it atomic? this goes for all 5 of the replies (one of you did not downvote discussion, I'm guessing that person is the only one not from the US). How does mysql unwrite the deposit? There are always "parts" to everything. Two phase commit does not solve anything unless you have "undo the thing I did before the power was lost".

Please refer to source code to prove your argumentation.

Re: Consistency is Consistently Undervalued

#84

Post by eternalban » Sat, Sep 17, 2016, 4:42 PM UTC

knucklesandwich wrote: post #82

vidarh wrote: post #68

Earlier quoted context omitted.

> You can't do this in general, because we often want to build applications where the user expects their actions won't be reordered. The key being they don't expect their actions to be reordered. In most systems this is trivially handled by logging event time as seen from the users system and ensure the user sees a consistent view of their own operations . There are exceptions, naturally, but the point is not that yo…

> Your privacy settings is a local concern to your account, it most certainly does not need to be globally consistent. All you need to do to avoid problems with that is to ensure consistency in routing requests for a given session, and enforce local consistency of ordering within a given user account or session depending on requirements. I think you're misunderstanding what the problem is here. I want global consiste…

> I want two party consistency

aka snapshot consistency.

(Thoroughly enjoyed your "the user is sort of the unstated extra party in a distributed system", btw.)

Re: Consistency is Consistently Undervalued

#85

Post by JoachimSchipper » Sat, Sep 17, 2016, 5:08 PM UTC

matt_wulfeck wrote: post #77

Maybe I'm crazy, but I never see atomic libraries that are called like this: bank_account2.deposit(amount) bank_account1.deposit(amount) Isn't this kind of thing always called in some atomic batch operation? transact.batch([ account[a] = -8, account[b] = 8 ]).submit()

Yes; "with atomic...:" seems to be pseudo-syntax.

Re: Consistency is Consistently Undervalued

#86

Post by marknadal » Sat, Sep 17, 2016, 5:16 PM UTC

Disclaimer: I work on distributed systems and have spoken around the world on them, so I am very biased.

I think something a lot of people miss is that the universe itself is not strongly consistent. This is Einstein's theory of relativity. It fundamentally takes time for information to travel.

So if strong consistency is not viable, even at a physics level, what can we do instead? That is why our team here at http://gun.js.org/ believe in CRDTs. What are CRDTs? They are data structures that are mathematically proven to produce the same results on retries - so even if the power goes out, or the network fails, you can safely re-attempt the update.

This means you fundamentally don't need "transactions" or any of these jargon words that people often throw out. Sadly CRDTs are becoming another one of those jargons, despite how simple they are in reality.

Re: Consistency is Consistently Undervalued

#87

Post by nickpsecurity » Sat, Sep 17, 2016, 5:30 PM UTC

ddorian43 wrote: post #55

amalcon wrote: post #52

Earlier quoted context omitted.

BerkeleyDB is great. I'm honestly surprised it doesn't see more use. It's right in the sweet spot for software that doesn't quite need a relational model, and that seems to be a lot of software.

Check out lmdb for a better alternative to it.

How did I not know about this awesome-looking tool? Thanks for the mention! Wikipedia link for others:

https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Databa...

Re: Consistency is Consistently Undervalued

#88

Post by wpietri » Sat, Sep 17, 2016, 5:33 PM UTC

vidarh wrote: post #58

kahirsch wrote: post #48

Earlier quoted context omitted.

As you say, the low-level operations in banking do provide atomicity, which are often lacking in distributed databases. Also, the banking system does have the system of compensating transactions (e.g. returned checks) when constraints are violated, as well as an extensive system of auditing to maintain consistency. Do most distributed systems have the ability to detect inconsistency? How do they manage to correct pro…

> Do most distributed systems have the ability to detect inconsistency? They do if you build them to have them. Your bank account works without requiring consistency because every operation is commutative: You never do anything that says "set account to value x". You have operations like "deposit x" and "withdraw x". For operations that require you move money from one account to another, you need to guarantee that bo…

I don't have much to add to your excellent post; I just wanted to underline this:

> Log events durably first, and then act on the events.

Nothing has changed my view of how to build systems more than the log-then-act paradigm. So many problems go away when I start from that perspective.

I think it took me so long to see it because so many common tools work against it. But those tools were built in an era when most systems had "the database", a single repository of everything. That paradigm is very slowly dying, and I look forward to seeing what tools we get when the distributed-systems paradigm becomes dominant.

Re: Consistency is Consistently Undervalued

#89

Post by hyc_symas » Sat, Sep 17, 2016, 5:34 PM UTC

Negitivefrags wrote: post #37

brandur wrote: post #30

Earlier quoted context omitted.

> It is very important that people shed this idea that only SQL databases can have strong consistency, that's for sure. Why? In practice, people are going to be building things using the same common paths: Postgres, Mongo, MySQL, Redis, etc., and from a purely pragmatic standpoint, the SQL databases that you're likely to use come with strong ACID guarantees while the NoSQL databases don't. If you can point me to a No…

Berkeley DB. I use it in production for a game that supports over 100,000 concurrent users and it's been around for a very long time. It's fully ACID by default but you get the ability to drop elements of ACID in exchange for greater performance in a per table or transaction level. It also supports two different isolation approaches on a per table level. Locking or MVCC (like postgres). It's a really great NoSQL data…

Berkeley DB was great, but Oracle killed it by changing the license to AGPL. That change meant many projects that use it couldn't update to the latest version without violating the license.

LMDB has taken its place in a lot of projects that used to use Berkeley.

Re: Consistency is Consistently Undervalued

#90

Post by smallnamespace » Sat, Sep 17, 2016, 5:36 PM UTC

bullen wrote: post #83

matthewmacleod wrote: post #13

Earlier quoted context omitted.

I don't mean to be unkind, but is this meant to be a parody? There are no "parts of a transaction" because a transaction is definitionally atomic. Transactional file modification is a fairly tricky problem, and I'd be surprised if you'd actually implemented a safe system in that manner. What's certain though is that spinning up MySQL or Postgres and using it to store simple records is essentially a zero-cost setup ta…

How is it atomic? this goes for all 5 of the replies (one of you did not downvote discussion, I'm guessing that person is the only one not from the US). How does mysql unwrite the deposit? There are always "parts" to everything. Two phase commit does not solve anything unless you have "undo the thing I did before the power was lost". Please refer to source code to prove your argumentation.

> How does mysql unwrite the deposit?

Yes, the database either reverses the changes.

https://en.wikipedia.org/wiki/Transaction_log

    If, after a start, the database is found in an
    inconsistent state or not been shut down properly, the
    database management system reviews the database logs for
    uncommitted transactions and rolls back the changes made
    by these transactions. Additionally, all transactions that
    are already committed but whose changes were not yet
    materialized in the database are re-applied. Both are done
    to ensure atomicity and durability of transactions.