Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen: NATS 2.12.1

jepsen.io

Post reply on HN
Print view View original

151–160 of 176 posts

Re: Jepsen: NATS 2.12.1

#151

Post by embedding-shape »

shikhar wrote: post #97
embedding-shape wrote: post #88

Earlier quoted context omitted.

Have you tried running Jepsen against it?

We do deterministic simulation testing https://s2.dev/blog/dst https://s2.dev/blog/linearizability We have also adopted Antithesis for a more thorough DST environment, and plan to do more with it. One day we will engage Kyle to Jepsen, too. I'm not sure when though.

I guess that's better than nothing. But now I'm unsure what your original comment was about, if your project doesn't use Jepsen for testing to "prove" it works fine, how is your project relevant to bring up on a submission about a Jepsen test of some other software?

If everyone who was making a database/message queue/whatever distributed system shared their projects on every Jepsen submission, we'd never have any discussions about the actual software in question.

Re: Jepsen: NATS 2.12.1

#152

Post by staticassertion »

awesome_dude wrote: post #62
stmw wrote: post #31

Every time someone builds one of these things and skips over "overcomplicated theory", aphyr destroys them. At this point, I wonder if we could train an AI to look over a project's documentation, and predict whether it's likely to lose commmitted writes just based on the marketing / technical claims. We probably can.

/me strokes my long grey beard and nods People always think "theory is overrated" or "hacking is better than having a school education" And then proceed to shoot themselves in the foot with "workarounds" that break well known, well documented, well traversed problem spaces

I don't have a "school education" and I know plenty of theory, I certainly have read the papers cited in this test.

Re: Jepsen: NATS 2.12.1

#153

Post by esafak »

63stack wrote: post #146
PeterCorless wrote: post #128

Earlier quoted context omitted.

You can have DeepWiki literally scan the source code and tell you: > 2. Delayed Sync Mode (Default) > In the default mode, writes are batched and marked with needSync = true for later synchronization filestore.go:7093-7097 . The actual sync happens during the next syncBlocks() execution. However, if you read DeepWiki's conclusion, it is far more optimistic than what Aphyr uncovered in real-world testing. > Durability…

and your point is ...?

You can DIY without aphyr.

Re: Jepsen: NATS 2.12.1

#154

Post by [deleted] »

awesome_dude wrote: post #62
stmw wrote: post #31

Every time someone builds one of these things and skips over "overcomplicated theory", aphyr destroys them. At this point, I wonder if we could train an AI to look over a project's documentation, and predict whether it's likely to lose commmitted writes just based on the marketing / technical claims. We probably can.

/me strokes my long grey beard and nods People always think "theory is overrated" or "hacking is better than having a school education" And then proceed to shoot themselves in the foot with "workarounds" that break well known, well documented, well traversed problem spaces

[deleted]

Re: Jepsen: NATS 2.12.1

#155

Post by mzl »

staticassertion wrote: post #152
awesome_dude wrote: post #62

Earlier quoted context omitted.

/me strokes my long grey beard and nods People always think "theory is overrated" or "hacking is better than having a school education" And then proceed to shoot themselves in the foot with "workarounds" that break well known, well documented, well traversed problem spaces

I don't have a "school education" and I know plenty of theory, I certainly have read the papers cited in this test.

You might not have a school education, but you have educated yourself. It is unfortunately common to hear people complain that the theory one learns in school (or by determined self-study) is useless, which I think is what the geybeard comment you replied to intends to say.

Re: Jepsen: NATS 2.12.1

#157

Post by staticassertion »

63stack wrote: post #146
PeterCorless wrote: post #128

Earlier quoted context omitted.

You can have DeepWiki literally scan the source code and tell you: > 2. Delayed Sync Mode (Default) > In the default mode, writes are batched and marked with needSync = true for later synchronization filestore.go:7093-7097 . The actual sync happens during the next syncBlocks() execution. However, if you read DeepWiki's conclusion, it is far more optimistic than what Aphyr uncovered in real-world testing. > Durability…

and your point is ...?

I don't think they were making a point. Someone suggested using an LLM for this, someone then responded by using an LLM for it.

What you draw from that seems entirely up to you. They don't seem to be making any claims or implying anything by doing so, just showing the result.

Re: Jepsen: NATS 2.12.1

#158

Post by [deleted] »

aaronbwebber wrote: post #18
merb wrote: post #5

> 3.4 Lazy fsync by Default Why? Why do some databases do that? To have better performance in benchmarks? It’s not like that it’s ok to do that if you have a better default or at least write a lot about it. But especially when you run stuff in a small cluster you get bitten by stuff like that.

It's not just better performance on latency benchmarks, it likely improves throughput as well because the writes will be batched together. Many applications do not require true durability and it is likely that many applications benefit from lazy fsync. Whether it should be the default is a lot more questionable though.

[deleted]

Re: Jepsen: NATS 2.12.1

#160

Post by plandis »

jwr wrote: post #148

For anyone dealing with databases, and especially distributed databases, I highly recommend reading the Jepsen page on consistency models: https://jepsen.io/consistency/models It provides a dictionary of terms that we can use to have educated discussions, rather than throwing around terms like "ACID".

I love that resource and reference it fairly frequently.

There is also this [1] which Aphyr collabed on which you might find interesting if you haven’t seen it yet.

[1] https://antithesis.com/resources/reliability_glossary/

Post reply on HN