Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Keeping CALM: when distributed consistency is easy

blog.acolyer.org

Post reply on HN
Print view View original

41–48 of 48 posts

Re: Keeping CALM: when distributed consistency is easy

#41

Post by mpweiher »

DanielBMarkham wrote: post #40
mpweiher wrote: post #38

Earlier quoted context omitted.

> OO and FP are just different flavors. Yes, current incarnations are really just variations of the call/return architectural style. Quite limited, actually. > Pure FP takes you places like this essay. Bold statement. OO has taken us to...let's see... all of personal computing . And somewhat smaller, but related, to Multi Version Concurrency Control: http://publications.csail.mit.edu/lcs/pubs/pdf/MIT-LCS-TR-20... > c…

I think we're mixing some things up here. If I have added to that confusion, I apologize. Yes, OO is great. It's all of modern computing. Love me some abstraction levels -- as long as they actually abstract things. Any layer that I spend more time screwing around with other than making things people want is a net negative. And there's my point: I'm not talking about the mechanics of FP or OO. I'm also not talking abo…

> At the end, it's all Category Theory.

Hmm. I thought it was all binary machine instructions. Or was it all NAND gates? Or everything is an object? Or everything is a list? Everything is a file?

Reductionist revelations are fun, and they also have some utility. But we have to be careful, because we tend to latch on to one of them and think that this is the one to rule them all. For different instances of "this" depending on person and time.

In reality it is all much more muddled:

https://www.infoq.com/presentations/functional-pros-cons/

> Whatever we put on top of the math to help us reason

The math is another one of those magic sprinkles you put on top of the NAND gates. If that's the one that gets you going, good for you! And it certainly has its uses. But don't get carried away.

> half-assed essays everyday ... "I learned OO and now I think of everything as objects"

Funny, I don't see any of those these days, maybe you're thinking of the 90s? Instead, I see a ton of blog posts claiming that "everything is category theory". ¯\_(ツ)_/¯. Very often, they then take an interesting problem, claim that it's really this other problem solved elegantly by FP. By stripping away all the properties of the problem that made it interesting and useful. But at least now it typechecks. Sigh.

Anyway...

> Love me some abstraction levels -- as long as they actually abstract things.

Yes. This is important. Really, really important. In my current thinking, there are currently three levels of this: just indirection, compression, and actual abstraction.

A lot of people think they've contributed abstraction when they're really just adding indirection. When you find an abstraction, it really is magic, but those are fairly rare. So you typically want to keep it as direct as possible, while trying to compress (trading off with directness as appropriate). The compression can point you towards an abstraction:

Refactoring Towards Language

https://blog.metaobject.com/2018/11/refactoring-towards-lang...

Another point is that the point of OO is not the program, it is the system created by the program. That is at the same time its greatest strength, and its greatest weakness, because it means that in order to build those systems, the program that constructs the system has to based on "side effects". But that's really a limitation of our call/return architectural pattern:

Thesis: Architecture Oriented Programming is what Object Oriented Programming wanted to be. It talks about instances and systems rather than classes and programs. It also lays a solid foundation for meta programming (connectors). -- https://twitter.com/mpweiher/status/1100283251531411457

So the systems are great (or can be, if you have a great systems builder), but the way we are forced to build them is pretty bad. FP tends to recognise the latter but not the former. See also:

Why Architecture Oriented Programming Matters

https://blog.metaobject.com/2019/02/why-architecture-oriente...

Re: Keeping CALM: when distributed consistency is easy

#42

Post by agentultra »

nickpsecurity wrote: post #33
deegles wrote: post #17

Earlier quoted context omitted.

What's a good small project + stack for someone to start with formal methods and/or program synthesis?

Formal verification is really hard because you have to use mathematical proof to show that a piece of software does what it should on all inputs and paths. You have to formalize the software, the desired goals, and prove a connection. Typically, you have to do this for software written in languages that aren't designed for verification. That makes it even harder. This led me to recommend people start with so-called l…

Nice list!

For proofs I've been finding the leap into dependent-type theory to be very productive as many provers in this realm also double as practical programming languages allowing the possibility of shipping proof-carrying code.

My two favorites in this area are Lean [0] and Agda [1], the latter of which recently received a nice introductory text from Phil Wadler and Wen Kokke [2].

[0] https://leanprover.github.io/ [1] https://wiki.portal.chalmers.se/agda/pmwiki.php [2] https://plfa.github.io/

Re: Keeping CALM: when distributed consistency is easy

#43

Post by oggy »

sagichmal wrote: post #34
oggy wrote: post #9

Earlier quoted context omitted.

That's an excellent explanation, except for couple of points which I think can be misinterpreted. A distributed system with consensus will in practice provide higher availability than a single-node system, because it provides fault-tolerance. In fact, fault-tolerance is the primary point of using (non-Byzantine) consensus. But you are absolutely right that a distributed system using consensus has worse availability t…

> A distributed system with consensus will in practice provide higher availability than a single-node system, because it provides fault-tolerance. I'm not sure this is true. It protects against one class of fault (node failure) but opens you up to another (network failure). As a distsys engineer I am increasingly convinced that fault tolerance is not a good selling point for distribution, the fallacies of distributed…

That's not really true. With a single, non-replicated server, you're also very much exposed to network failures. If the server's connection goes down, you're screwed. Compare this to a Google replication setup of 2 East Cost, 2 West Coast, and 1 central USA server. The client must only be able to reach two out of three data centers (and they have to be able to communicate with each other). That sounds much more resilient to me - and I guess Google agrees, since they deploy the setup.

Re: Keeping CALM: when distributed consistency is easy

#44

Post by oggy »

zzzcpan wrote: post #10
oggy wrote: post #9

Earlier quoted context omitted.

That's an excellent explanation, except for couple of points which I think can be misinterpreted. A distributed system with consensus will in practice provide higher availability than a single-node system, because it provides fault-tolerance. In fact, fault-tolerance is the primary point of using (non-Byzantine) consensus. But you are absolutely right that a distributed system using consensus has worse availability t…

> when using a system "with consensus" there is often no need to actually invoke consensus on the read side of the system, in which case you don't have to pay the throughput and latency penalties. Doesn't matter if the system is designed for faster reads. There is still coordination, you still pay coordination overhead, including throughput and latency penalties. Without coordination, for example, you can have some p…

You're comparing apples to oranges. If read coordination can be avoided (and it often can, even in systems with consensus), whether you stick a cache on the client or not is completely orthogonal to whether the system uses consensus for write operations.

Re: Keeping CALM: when distributed consistency is easy

#45

Post by zzzcpan »

oggy wrote: post #44
zzzcpan wrote: post #10

Earlier quoted context omitted.

> when using a system "with consensus" there is often no need to actually invoke consensus on the read side of the system, in which case you don't have to pay the throughput and latency penalties. Doesn't matter if the system is designed for faster reads. There is still coordination, you still pay coordination overhead, including throughput and latency penalties. Without coordination, for example, you can have some p…

You're comparing apples to oranges. If read coordination can be avoided (and it often can, even in systems with consensus), whether you stick a cache on the client or not is completely orthogonal to whether the system uses consensus for write operations.

Think about it, if you read from a replica that is partitioned from the rest of the system and there is no coordination, how would the replica or the client know that the value replica returns is too old and therefore breaks strong consistency guarantee?

Re: Keeping CALM: when distributed consistency is easy

#46

Post by dasvadanya »

>"Rather than micro-optimize protocols to protect race conditions in procedural code, modern distributed systems creativity often involves minimizing the use of such protocols."

Commerical databases 20 years ago had "shared nothing" designs precisely to avoid locks. The term "embarrassingly parallel" was introduced in the 1980s! The technique of multi-version concurrency control is widely used in databases and was first introduced in the 1980s. Maybe the authors forgot.

Re: Keeping CALM: when distributed consistency is easy

#47

Post by oggy »

zzzcpan wrote: post #45
oggy wrote: post #44

Earlier quoted context omitted.

You're comparing apples to oranges. If read coordination can be avoided (and it often can, even in systems with consensus), whether you stick a cache on the client or not is completely orthogonal to whether the system uses consensus for write operations.

Think about it, if you read from a replica that is partitioned from the rest of the system and there is no coordination, how would the replica or the client know that the value replica returns is too old and therefore breaks strong consistency guarantee?

No, you don't get linearizability of all operations, you might not even read your earlier writes - but the whole point is that you sometimes don't need these guarantees for reads. You get a consistent snapshot read, and that's often good enough. You can get an idea of how recent the snapshot is based on timestamps, but "recent" is hard to define in a distributed system.

Re: Keeping CALM: when distributed consistency is easy

#48

Post by zzzcpan »

oggy wrote: post #47
zzzcpan wrote: post #45

Earlier quoted context omitted.

Think about it, if you read from a replica that is partitioned from the rest of the system and there is no coordination, how would the replica or the client know that the value replica returns is too old and therefore breaks strong consistency guarantee?

No, you don't get linearizability of all operations, you might not even read your earlier writes - but the whole point is that you sometimes don't need these guarantees for reads. You get a consistent snapshot read, and that's often good enough. You can get an idea of how recent the snapshot is based on timestamps, but "recent" is hard to define in a distributed system.

> but the whole point is that you sometimes don't need these guarantees for reads.

If that was your point, than sure. If you drop consistency, you can drop coordination too. But typically people expect reads to be consistent in consensus based systems, which requires coordination.

Post reply on HN