Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

50 years later, is two-phase locking the best we can do?

concurrencyfreaks.blogspot.com

Post reply on HN
Print view View original

81–87 of 87 posts

Re: 50 years later, is two-phase locking the best we can do?

#81

Post by matthewsinclair »

ris wrote: post #62
samsquire wrote: post #35

I'm a beginner in this topic and I find this topic interesting. I really want there to be an easy-to-deploy consistency solution. If I have a distributed microservice architecture and I want to keep multiple datastores in synchronization or "consistent" what's the industry best practice? A few days ago I was trying to solve the inconsistency problem with "settled timestamps" which is a kind of multiversioning idea ex…

> If I have a distributed microservice architecture and I want to keep multiple datastores in synchronization or "consistent" what's the industry best practice? Not to use a distributed microservice architecture.

So much this.

I was part of an engineering team that built a payments switch from scratch in the early oughts.

We built it in Java, on commodity hardware and OS, on top of our own replicated, stateful, distributed computing platform.

This was a bonkers thing to do then, pre-cloud. It’s probably still a bonkers thing to do _today_.

Anyway, we did a lot of work with 2PC and other consensus mechanisms and came to the conclusion they 2PC wasn’t up to scratch for what we needed (it’s actually provably less than ideal).

We ended up building (again, from scratch) one of the earliest (that I know of) implementations of the Virtual Synchrony protocol. VS has some robust maths behind it that you can use to make some stronger consistency claims than you can with 2PC. These are important when you’re dealing with interbank settlements for payments switching.

If we started again today I’d say that we might use something like Raft, but I’ve been away from the space for ~20 years now so I’m not entirely sure.

However, one thing I do know is this: distributed consensus is Very Hard™ to get right.

If the answer to your question involves any kind of multiple master implementation using distributed consensus I can unequivocally guarantee that (other than in a few very specific circumstances that you almost certainly do not have) you’re asking the wrong question.

Re: 50 years later, is two-phase locking the best we can do?

#83

Post by anticensor »

rawgabbit wrote: post #68
zaphirplane wrote: post #37

Earlier quoted context omitted.

IIUC, That’s a micro service way, missing cancellations and rollback and timeouts What happens if the chef can’t fulfill the meal, the customer leaves cause the order is taking long … etc Most real life will not charge the customer is the chef can’t make the meal and cancel the cooking if the customer leaves The analogy maybe going far ;)

True. If the customer leaves in mid process, we would have to issue a reversal transaction. Eat the financial loss, dispose of the food, and void the order the ticket.

Or do it like a fast food restaurant: take the order, take the money, finalise the transaction but not commit (irreversible from that point on) then cook the food then commit: if the customer leaves before taking the meal, serve it to the next customer ordering the same meal.

Re: 50 years later, is two-phase locking the best we can do?

#84

Post by nine_k »

niederman wrote: post #76
sfifs wrote: post #59

Earlier quoted context omitted.

Isn't bitcoin actually eventual consensus determined by increasing unlikelihood of finding a conflicting sequence of hashes?

Yes, although if the consistency is good enough for _payments infrastructure_, it's probably good enough for just about everything.

If a particualr mechanic part is good enough for a _space ship_, it's probably good enough for just about everything.

But its price may be prohibitive.

Re: 50 years later, is two-phase locking the best we can do?

#85

Post by rusk »

nine_k wrote: post #75
rusk wrote: post #51

Earlier quoted context omitted.

Bitcoin is an interesting exception to the above. Issue is latency.

Latency might be OK. The true price of trustless consensus is proof of work, that is, energy.

> Latency might be okay.

Nope. Not for real-time gaming which is the topic at hand. Certainly not at scale. Certainly not at the kind of volume of events that gaming produces.

Re: 50 years later, is two-phase locking the best we can do?

#86

Post by ngcc_hk »

Forever it is. And never perfect it is.

The issue is if your first locking message gone how do you know it is not the reply message that is lost. For simple one just go ahead and deal with conflict later like GitHub and Dropbox etc. for database good luck. For bank.

Re: 50 years later, is two-phase locking the best we can do?

#87

Post by [deleted] »

samsquire wrote: post #58
theptip wrote: post #55

Earlier quoted context omitted.

Look at the Raft protocol. In general you’d rather not integrate at the protocol layer; it’s standard to use a consistent store like etcd that implements Raft for the coordination/data that needs to be serializable. (Kubernetes uses etcd so it scales pretty well for a strongly-consistent k/v store.) You said “multiple datastores” so I’m assuming you have heterogenous data and something like CockroachDB isn’t an optio…

Thanks for your reply. I've experimented with a toy Raft implementation but I haven't Jepsen tested that and it's incomplete I did write a Jepsen test for a different eventually consistent protocol which understandably fails the linearizability test because I'm still learning - eventually consistent is not linearizable. https://GitHub.com/samsquire/eventually-consistent-mesh I want to have my cake and eat it too. Sca…

[deleted]
Post reply on HN