Live data from Hacker News

Jepsen: Radix DLT 1.0-Beta.35.1

jepsen.io

41–50 of 78 posts

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#41

OH YEAH JEPSEN IS TEARING APART CRYPTO!!!! Aphyr is the best. No matter your fave distributed tech and all it's CAP-don't-matter stuff, he shows that... CAP does very much matter and it's really hard. Cassandra? Kafka? MongoDB (bwahahahah)? It's all got edge cases. He should be getting paid a million bucks a year by various auditing/accounting firms and the FTC/SEC for validating crypto claims. It would be a massive…

Jepsen is most certainly not "tearing apart crypto" nor is that even remotely the intention of this post. From the very first paragraph: >"This work was funded by Radix Tokens (Jersey) Limited, and conducted in collaboration with RDX Works Ltd ..." The post also links to RDX Works Ltd's blog post on this collaboration. Also in the first paragraph.

That's the best part: they paid him to throw such savage shade on their blockchain. One transaction per second, one million transactions per second, what's the difference? It's just the roadmap, man.

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#42
post #39
post #38

Earlier quoted context omitted.

Although I agree with your point, on language and its ambiguity, I would argue that is a different claim of the one you made above, and that I replied to. When he demonstrated that Riak was dropping 30-70% of writes, even with the strongest consistency settings, or that Mongo had multiple scenarios of data loss, we are not talking about the subtleties of the English idiom

But the contrived nature of Jepsen tests is so different than the real world. In the real world no system behaves exactly the same as it was designed to behave; the real world has cosmic rays, earthquakes and everything in between. So no statement about a software system can ever be simply accepted as a fundamental fact and so to prove that software systems break is misleading for what most users need to understand a…

> But the contrived nature of Jepsen tests is so different than the real world.

The real world is significantly more "contrived" than anything Jepsen can come up with.

Transactions timing out, Nodes losing transactions even after they've been acknowledged etc.? None of this is contrived.

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#43
post #41

Earlier quoted context omitted.

Jepsen is most certainly not "tearing apart crypto" nor is that even remotely the intention of this post. From the very first paragraph: >"This work was funded by Radix Tokens (Jersey) Limited, and conducted in collaboration with RDX Works Ltd ..." The post also links to RDX Works Ltd's blog post on this collaboration. Also in the first paragraph.

That's the best part: they paid him to throw such savage shade on their blockchain. One transaction per second, one million transactions per second, what's the difference? It's just the roadmap, man.

Makes you wonder why they ended the collaboration in November and didn't continue to retain him to validate that the new stuff lives up to its claims, doesn't it?

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#44
post #40
post #39

Earlier quoted context omitted.

But the contrived nature of Jepsen tests is so different than the real world. In the real world no system behaves exactly the same as it was designed to behave; the real world has cosmic rays, earthquakes and everything in between. So no statement about a software system can ever be simply accepted as a fundamental fact and so to prove that software systems break is misleading for what most users need to understand a…

We do know that some databases fare much better than others and that’s useful to many. In your analogy it would be many builders claiming their buildings to be earthquake proof with only some actually being it. Thanks to Jepsen, customers know, where to buy.

I'm genuinely curious to hear which database is the winner from your perspective based on Jepsen's insight?

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#45
post #39
post #38

Earlier quoted context omitted.

Although I agree with your point, on language and its ambiguity, I would argue that is a different claim of the one you made above, and that I replied to. When he demonstrated that Riak was dropping 30-70% of writes, even with the strongest consistency settings, or that Mongo had multiple scenarios of data loss, we are not talking about the subtleties of the English idiom

But the contrived nature of Jepsen tests is so different than the real world. In the real world no system behaves exactly the same as it was designed to behave; the real world has cosmic rays, earthquakes and everything in between. So no statement about a software system can ever be simply accepted as a fundamental fact and so to prove that software systems break is misleading for what most users need to understand a…

Jepsen is all about verifying claims and communicating limitations. Companies hire them for this as due diligence. They fix real world bugs based on the results.

> what most users need to understand about said systems

The target audience of these reports are the system builders and the software engineers building services on top of them; not end-users consuming higher level services.

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#46
post #39
post #38

Earlier quoted context omitted.

Although I agree with your point, on language and its ambiguity, I would argue that is a different claim of the one you made above, and that I replied to. When he demonstrated that Riak was dropping 30-70% of writes, even with the strongest consistency settings, or that Mongo had multiple scenarios of data loss, we are not talking about the subtleties of the English idiom

But the contrived nature of Jepsen tests is so different than the real world. In the real world no system behaves exactly the same as it was designed to behave; the real world has cosmic rays, earthquakes and everything in between. So no statement about a software system can ever be simply accepted as a fundamental fact and so to prove that software systems break is misleading for what most users need to understand a…

You make some reasonable points, but I would say a couple things here:

- If we're sticking with your example further up-thread, you'd buy a house that was advertised as "can withstand a 4.0 magnitude earthquake", that then failed when subjected to a 4.0 magnitude earthquake.

- 4.0 magnitude earthquakes happen all of the time [0].

More or less what I'm saying is, engineers generally don't think DBs lose data, and when they start coming up with ways that might happen (node failure, network failure, clock desync), distributed DBs assure them with algorithms and configuration knobs. Aphyr puts those assurances to the test, which is so, so valuable to us all.

It's also worth saying that this space is pretty technically complicated. All the DB engineers I know use some form of Jepsen-style testing (or Jepsen itself) because it's amazingly great.

[0]: https://research.google/pubs/pub40801/

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#47
post #41

Earlier quoted context omitted.

That's the best part: they paid him to throw such savage shade on their blockchain. One transaction per second, one million transactions per second, what's the difference? It's just the roadmap, man.

Makes you wonder why they ended the collaboration in November and didn't continue to retain him to validate that the new stuff lives up to its claims, doesn't it?

It's not a secret at all. The response from the Radix CEO: "We re-used the part of his testing harness, and then re-created the other critical tests to determine if the errors detected where still present. He is fantastically expensive and booked 3-6 months in advance on average. We'll definite re-deploy this kind of testing again, but we'll save it for another bigger release, rather than a patch where the identified errors can be tested against."

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#48

Earlier quoted context omitted.

I live outside the United States and have never seen mm mean anything other than millimeter

Meanwhile I live in the UK and it's completely standard for monetary quantities.

I’ve seen that use for sure. Perhaps a little telling we’re talking about blockchain tech and that’s what people are using. Is it really about a fast distributed transaction log, or… is it about the money first.

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#49
post #35
post #34

Earlier quoted context omitted.

That is not what Jepsen demonstrates. Instead it shows systems making claims that are not supported by their implementation and/or algorithms. If you want to take on your claim/suggestion then those systems just have to stop making the claims he is testing for. PS: There is also another trend, that is System claiming they fixed the issues Jepsen finds, without submitting themselves again for analysis...but I digress…

I respectfully disagree. Human language and characteristics around distributed data stores in the real world have inherent ambiguity implicit in them that Jepsen in my view pretends isn't the case. Here's an analogy that may help me communicate how I feel since I realize my message is not landing: Let's say I'm buying a condo in the San Francisco Bay area. And let's say the building that I'm looking to buy in adverti…

> I respectfully disagree. Human language and characteristics around distributed data stores in the real world have inherent ambiguity implicit in them that Jepsen in my view pretends isn't the case.

We've been doing distributed systems for over 50 years now. There's nothing ambiguous in either the language Jepsen uses or the claims he is examining. The crypto shills chose to pretend the language is ambiguous and invent their own definitions on purpose.

Including the ridiculous "nah, everyone understands that when we speak in present tense it means we mean some unspecified point in a nebulous future".

Re: Jepsen: Radix DLT 1.0-Beta.35.1

#50
post #47

Earlier quoted context omitted.

Makes you wonder why they ended the collaboration in November and didn't continue to retain him to validate that the new stuff lives up to its claims, doesn't it?

It's not a secret at all. The response from the Radix CEO: "We re-used the part of his testing harness, and then re-created the other critical tests to determine if the errors detected where still present. He is fantastically expensive and booked 3-6 months in advance on average. We'll definite re-deploy this kind of testing again, but we'll save it for another bigger release, rather than a patch where the identified…

I look forward to reading that report!
Post reply on HN