Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen: NATS 2.12.1

jepsen.io

Post reply on HN
Print view View original

171–176 of 176 posts

Re: Jepsen: NATS 2.12.1

#171

Post by shikhar »

pdimitar wrote: post #134
shikhar wrote: post #81

If you are looking for a serverless alternative to JetStream, check out https://s2.dev Pros: unlimited streams with the durability of object storage – JetStream can only do a few K topics Cons: no consumer groups yet, it's on the agenda

I'm not seeing full self-hosting yet, and "Book a call" link is an instant nope for many techies. I understand that you need to make money. But you'll have to have a proper self-hosting offering with paid support as well before you're considered, at least by me. I'm not looking to have even more stuff in the cloud.

The cloud offering is self-serve, no need to get on a call at all. An open source, self-hosted option is in progress https://github.com/s2-streamstore/s2?tab=readme-ov-file#s2-l...

Re: Jepsen: NATS 2.12.1

#172

Post by shikhar »

embedding-shape wrote: post #151
shikhar wrote: post #97

Earlier quoted context omitted.

We do deterministic simulation testing https://s2.dev/blog/dst https://s2.dev/blog/linearizability We have also adopted Antithesis for a more thorough DST environment, and plan to do more with it. One day we will engage Kyle to Jepsen, too. I'm not sure when though.

I guess that's better than nothing. But now I'm unsure what your original comment was about, if your project doesn't use Jepsen for testing to "prove" it works fine, how is your project relevant to bring up on a submission about a Jepsen test of some other software? If everyone who was making a database/message queue/whatever distributed system shared their projects on every Jepsen submission, we'd never have any dis…

It seemed like the kind of Jepsen outcome where folks would be considering alternatives, but yeah maybe it was not appropriate to plug here.

Re: Jepsen: NATS 2.12.1

#173

Post by menaerus »

sreekanth850 wrote: post #120
akshayshah wrote: post #117

Earlier quoted context omitted.

No. Redpanda has made a lot of noise about this over the years [0], and Confluent's Jack Vanlightly has responded in a fair bit of detail [1]. [0]: https://www.redpanda.com/blog/why-fsync-is-needed-for-data-s... [1]: https://jack-vanlightly.com/blog/2023/4/24/why-apache-kafka-...

I think all modern system even scylla db do commit batch no fsync on every write, you either need throughput or durability both cannot exist together. Only thing what redpanda claim is you have to do replication before fsync so your data is not lost if the written node is dead due to a power failure. this is how scylla and cassandra works, if iam not wrong, so even if a node dead before the batch fsync, replication w…

Similarly in regular SQL systems, the same is achieved by fsyncing to WAL.

Re: Jepsen: NATS 2.12.1

#174

Post by otabdeveloper4 »

staticassertion wrote: post #165
otabdeveloper4 wrote: post #149

Earlier quoted context omitted.

> Many applications do not require true durability Pretty much no application requires true durability.

Maybe what's confusing here is "true durability" but most people want to know that when data is committed that they can reason about the durability of that data using something like a basic MTBF formula - that is, your durability is "X computers of Y total have to fail at the same time, at which point N data loss occurs". They expect that as the number Y goes up, X goes up too. When your system doesn't do things like…

For the vast majority of applications a rare event of data loss is no big deal and even expected.

Re: Jepsen: NATS 2.12.1

#175

Post by staticassertion »

otabdeveloper4 wrote: post #174
staticassertion wrote: post #165

Earlier quoted context omitted.

Maybe what's confusing here is "true durability" but most people want to know that when data is committed that they can reason about the durability of that data using something like a basic MTBF formula - that is, your durability is "X computers of Y total have to fail at the same time, at which point N data loss occurs". They expect that as the number Y goes up, X goes up too. When your system doesn't do things like…

For the vast majority of applications a rare event of data loss is no big deal and even expected.

I think you're still not getting my point. Yes, a rare event of data loss may not be a big deal. What is a big deal is being able to reason about how rare that event is. When you have durable raft you can reason by using straightforward MTBF calculations. When you don't, you can keep adding nodes but you can't use MTBF anymore because a single failure is actually sufficient to cause data loss.
Post reply on HN