Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The CAP theorem. The Bad, the Bad, & the Ugly

blog.dtornow.com

Post reply on HN
Print view View original

71–80 of 81 posts

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#71

Post by DandyDev »

harkinian wrote: post #67
justin_hancock wrote: post #33

Earlier quoted context omitted.

Disagree on latency, for certain classes of applications it really matters. Latency can be caused by a myriad of things, but if that thing is your means for cut over, you haven't failed to respond, you've failed to respond in a timely way. The telco space has strict latency requirements. If you have a failure which results in erratic behaviour on the network due to the database taking time to handle a node failure yo…

Put an SLA on the latency, and it becomes availability.

That’s a great way of putting it!

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#72

Post by gwbas1c »

filleokus wrote: post #26
valenterry wrote: post #24

Earlier quoted context omitted.

Sure. But the thing is: is there literally not a single component in that mainframe that cannot cause a single point of failure to the whole system? If so, I agree - in that case it would be similar to running two severs in the same AZ. Which provides a certain form of availability. Otherwise I disagree, because as soon as you have a single point of failure, your system is limited by the availability of that specific…

I've been "arguing" AGAINST mainframes on HN in the past, and after that I've seen many really cool presentations about them. One, quite commonly known, is that two separate mainframes can run completely in lock-step if they are less than ≈ 50 km / 30 mi apart. So if one explodes mid instruction it will still continue fine. As for a single system, I don't think there is any single point of failure, you can swap cpu's…

More importantly, for a lot of the things that people build today, a mainframe is powerful enough to handle however large the load will get.

The ultra-scalable techniques that Google, Facebook, ect, use are great, but most applications do not need that kind of scalability. Throwing a little more money on a fancy mainframe is much, much cheaper than the programmer time needed to build a distributed system.

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#73

Post by valenterry »

samatman wrote: post #70
valenterry wrote: post #60

Earlier quoted context omitted.

That's not what this discussion is about. OP said: > as soon as you decided to distribute your system across a network you opted into a world where partition can happen, and you will have to give up consistency or availability. So we are talking in the context of distributed networks. Then: > Mainframes, though, provide consistency and availability by being unpartitionable except through use of a chainsaw. using the…

Your fire is just the chainsaw in the original post.

Yeah, that's exactly my point.

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#74

Post by deathanatos »

willsmith72 wrote: post #11

I don't find this position particularly helpful, unless you're talking to someone who takes CAP literally as a law and picks only 2. That's never been the case in discussions I've had. It's still extremely useful as a mental model for tradeoffs in designing a system. All it's telling you is there's a pendulum from availability to consistency and generally. The more you want of one, generally the less you get of the o…

> Has anyone in their company really argued that, "we should choose availability instead of consistency"? It's way more nuanced than that, and as far as I know, everyone knows that

… I'm usually experiencing the CP/AP split from the seat of a user, using the system. From where I sit … yes, it certainly feels like people are, somewhere, saying "we should choose availability", given the number of systems I've had to interact with that are trivially not CP.

Specifically, refer to https://jepsen.io/consistency — these are better, more specific terms, IMO; the number of systems that don't obey "Read Your Writes" (at the very bottom of the tree!) is pretty stark. Many Azure services, for example, are not read-your-writes; this means I'm perpetually wrapping things in loops that attempt to wait for the upstream system to become consistent, which is impossible to do in any manner that's foolproof, prior to moving on to the next API call (which would otherwise fail, if it depends on a write from the prior API call, but can't read it, because read-your-writes). Off the top of my head, I've seen empirical violations of read-your-writes in all of ARM, AAD, ACR. My latest container registry is also not read-your-writes.

S3 used to fail read-your-writes, in certain circumstances. (That have since been fixed.)

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#75

Post by j-pb »

klabb3 wrote: post #58
j-pb wrote: post #45

There is a theorem called CALM[1] (Consistency as Logical Monotonicity) that shows that systems are eventually consistent iff they are monotonic. I find that reasoning about consistency is much easier from that perspective, for example it immediately gives you an intuition on why individual CRDTs work. 1: https://arxiv.org/abs/1901.01930

Sounds interesting. Is there an intuitive definition of monotonicity for the purposes of distributed systems? I mean, I thought I knew what it means, but I’m not too sure now.

You never take something back. A grow set for example only ever grows. A monotonic counter only ever grows. Even a set where you can insert and delete once is like a set of counters that only ever grow (nothing: 0, added:1, removed:2).

Forgetting for example is fine, but deletion is not.

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#76

Post by YetAnotherNick »

jameshart wrote: post #59
YetAnotherNick wrote: post #32

Earlier quoted context omitted.

> The ‘network partitions are not optional’ interpretation of CAP is forgetting about the fact that non-distributed-systems are a thing. Not only that, a weird network partition is highly rare and I have never seen it. In most of the cases, there is only one network and either the server is up and connected to "the" network or is not connected to the network. And I believe engineers have tendency to overengineer for…

Minimum of three is generally for one of two reasons: 1) to ensure you can deliberately take one offline for maintenance and still have redundancy in case a single node goes offline 2) in some systems an odd number of nodes is needed to ensure no ties in leader elections or decision votes. Three is the smallest odd number that has any redundancy.

1) I have taken a node in maintenance mode many times deliberately by launching second node and draining first. Its not an issue at all with just one node. For redundancy, is there ever a case where single node could go down in practice. Nodes are basically EC2 instance and they could run for years without going down.

2) 1 is odd and could win the election;)

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#77

Post by jameshart »

YetAnotherNick wrote: post #76
jameshart wrote: post #59

Earlier quoted context omitted.

Minimum of three is generally for one of two reasons: 1) to ensure you can deliberately take one offline for maintenance and still have redundancy in case a single node goes offline 2) in some systems an odd number of nodes is needed to ensure no ties in leader elections or decision votes. Three is the smallest odd number that has any redundancy.

1) I have taken a node in maintenance mode many times deliberately by launching second node and draining first. Its not an issue at all with just one node. For redundancy, is there ever a case where single node could go down in practice. Nodes are basically EC2 instance and they could run for years without going down. 2) 1 is odd and could win the election;)

EC2 instances can be rebooted at any time. The underlying hardware they are on fails from time to time and they get moved.

Running three nodes is a rule of thumb, not a hard rule for minimally guaranteeing availability.

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#78

Post by _visgean »

12345hn6789 wrote: post #47
_visgean wrote: post #5

Earlier quoted context omitted.

I had an interview question about it, I tried to cite Kleppmann from the distributed systems book and the interview manager got quite offended.

care to elaborate? Not picking apart anything you said but am curious. I am slowly making my way through DDIA. Nothing he says is particularly egregious (so far).

the guy interviewing me just wanted a definition. I guess I should have started with that and then point out why its a bad concept.

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#79

Post by YetAnotherNick »

jameshart wrote: post #77
YetAnotherNick wrote: post #76

Earlier quoted context omitted.

1) I have taken a node in maintenance mode many times deliberately by launching second node and draining first. Its not an issue at all with just one node. For redundancy, is there ever a case where single node could go down in practice. Nodes are basically EC2 instance and they could run for years without going down. 2) 1 is odd and could win the election;)

EC2 instances can be rebooted at any time. The underlying hardware they are on fails from time to time and they get moved. Running three nodes is a rule of thumb, not a hard rule for minimally guaranteeing availability.

> EC2 instances can be rebooted at any time

No, they can't be rebooted any time. Where are you getting this information from?

Re: The CAP theorem. The Bad, the Bad, & the Ugly

#80

Post by jameshart »

YetAnotherNick wrote: post #79
jameshart wrote: post #77

Earlier quoted context omitted.

EC2 instances can be rebooted at any time. The underlying hardware they are on fails from time to time and they get moved. Running three nodes is a rule of thumb, not a hard rule for minimally guaranteeing availability.

> EC2 instances can be rebooted at any time No, they can't be rebooted any time. Where are you getting this information from?

https://repost.aws/knowledge-center/ec2-linux-degraded-hardw...

Note in particular:

> For instances that launched from an Amazon EC2 Auto Scaling group, the instance termination and replacement occur immediately

Post reply on HN