Distributed Systems Shibboleths
jolynch.github.io
Distributed Systems Shibboleths
1–10 of 72 posts
Re: Distributed Systems Shibboleths
#2[deleted]
Re: Distributed Systems Shibboleths
#3[deleted]
Re: Distributed Systems Shibboleths
#4I enjoyed reading that a lot.
> The main advantage of distributed transactions is that they make distributed systems look less distributed by choosing CP, but that inherently trades off availability!
This is true, but I suspect that its slightly missing the important thing about transactions. A transaction is an operation that takes the database from one Consistent (ACID "C", you can think about it as "legal under the business logic") state to another Consistent state. Linearizability (CAP "C") isn't enough to do that, because often changes in databases require "take from Bob and give to Alice", or "check Bob and Alice's balance and add the order", neither of which fit well into Linearizability's single-key definition of the world. Allowing developers to think about a stream of operations that moves the databases from one legal state to another is super powerful. The whole point is that it provides an abstraction that hides concurrency (ACID "I") and partial failure (ACID "A"). Saving developers from reasoning about those is a big win!
> I should also note that while Distributed Transactions might be a useful tool in building idempotency, simply wrapping a non idempotent operation (e.g. “add 1 to X”) in a transaction does not make it idempotent.
The OP is right that this isn't a panacea, especially where those transactions aren't idempotent. But transactions are a mechanism to implement idempotence ("insert order number 10 if it isn't there already"), and idempotence and ACID "C" can be really hard to achieve without transactions (or at least "I" and "A").
Transactions, CRDTs, and the CALM theorem are linked too. You can definitely have transactions in systems that aren't CAP "C" consistent, and still have them do legal things. The CALM theorem lays out one way to think about those, and CRDTs are a kind of object-oriented embodiment of that theory.
Re: Distributed Systems Shibboleths
#5Am I missing why a distributed lock is an impossibility? The problem stated is that a partitioned node can't know it has lost the lock, but this is only an issue if there is a way to lose the lock short of returning it.
Which I guess is to say: what difference is there between a lease with an infinite timeout unless manually returned, and a "lock"?
Certainly the system deadlocks under partition but I'm not sure why that makes this "impossible".
Re: Distributed Systems Shibboleths
#6> database vendors might try just a little harder to tell the truth...
Come on, you know that's not what's going to happen. If they notice at all, they'll just incorporate the magic phrases into their BS so you have to hunt harder for a real signal.
Re: Distributed Systems Shibboleths
#7This was a great post and covers many day to day topics that practitioners tend to hand wave over, especially as distributed systems are becoming more pervasive. Dare I say even some of the statements are becoming cliches.
The section on distributed transactions could have a little more nuance. Particularly the example about the counter where I suspect any system offering transactions also has a CAS operation. Additionally the benefit of a transaction system is that you can offer bounded counters where as an AP or “strong” EC (CRDTs) system cannot.
Re: Distributed Systems Shibboleths
#8Am I missing why a distributed lock is an impossibility? The problem stated is that a partitioned node can't know it has lost the lock, but this is only an issue if there is a way to lose the lock short of returning it. Which I guess is to say: what difference is there between a lease with an infinite timeout unless manually returned, and a "lock"? Certainly the system deadlocks under partition but I'm not sure why t…
replace "impossible" with "not really useful"
Re: Distributed Systems Shibboleths
#9Am I missing why a distributed lock is an impossibility? The problem stated is that a partitioned node can't know it has lost the lock, but this is only an issue if there is a way to lose the lock short of returning it. Which I guess is to say: what difference is there between a lease with an infinite timeout unless manually returned, and a "lock"? Certainly the system deadlocks under partition but I'm not sure why t…
I think it's possible to build such a lock, but if your lock will inevitably deadlock due to random failures it's probably not that useful.
Re: Distributed Systems Shibboleths
#10Am I missing why a distributed lock is an impossibility? The problem stated is that a partitioned node can't know it has lost the lock, but this is only an issue if there is a way to lose the lock short of returning it. Which I guess is to say: what difference is there between a lease with an infinite timeout unless manually returned, and a "lock"? Certainly the system deadlocks under partition but I'm not sure why t…
The short answer is because if the lock holder fails your other nodes have no way of knowing if the lock holder failed (consequence of FLP Impossibility result). If you set a timeout, then that’s a lease.
The long answer is to peel this onion for yourself and see where it leads. It’s a lot of fun.