Live data from Hacker News

Jepsen: NATS 2.12.1

jepsen.io

91–100 of 176 posts

Re: Jepsen: NATS 2.12.1

#91

Earlier quoted context omitted.

Not flushing on every write is a very common tradeoff of speed over durability. Filesystems, databases, all kinds of systems do this. They have some hacks to prevent it from corrupting the entire dataset, but lost writes are accepted. You can often prevent this by enabling an option or tuning a parameter. > I wouldn't trust a product that doesn't default to safest options This would make most products suck, and requi…

> Filesystems, databases, all kinds of systems do this. They have some hacks to prevent it from corrupting the entire dataset, but lost writes are accepted. Woah, those are _really_ strong claims. "Lost writes are accepted"? Assuming we are talking about "acknowledged writes", which the article is discussing, I don't think it's true that this is a common default for databases and filesystems. Perhaps databases or K/V…

All filesystems that I'm aware of don't sync to disk on every write by default, and you absolutely can lose data. You have to intentionally enable sync. And even then the disk can still lose the writes.

Most (all?) NoSQL solutions are also eventual-consistency by default which means they can lose data. That's how Mongo works. It syncs a journal every 30-100 ms, and it syncs full writes at a configurable delay. Mongo is terrible, but not because it behaves like a filesystem.

Note that this is not "bad", it's just different. Lots of people use these systems specifically because they need performance more than durability. There are other systems you can use if you need those guarantees.

Re: Jepsen: NATS 2.12.1

#93
post #85

Curious about the differences between content on aphyr.com/tags/jepsen and jepsen.io/analyses. I recently discovered aphyr.com and was excited about the potential insights!

Jepsen started as a personal blog series in nights and weekends; jepsen.io is when I started doing it professionally, about ten years ago.

Re: Jepsen: NATS 2.12.1

#94

Earlier quoted context omitted.

People overly beholden to tried and true 'known' way of addressing a problem space and not considering/belittling alternatives. Many of the things that have been most aggressively 'bitter lesson'ed in the last decade fall into this category.

Like this bug report? The things that have been "disrupted" haven't delivered - Blockchains are still a scam, Food delivery services are worse than before (Restaurants are worse off, the people making the deliveries are worse off), Taxis still needed to go back and vet drivers to ensure that they weren't fiends.

> Blockchains are still a scam

Did you actually look at the blockchain nodes implementation as of 2025 and what's in the roadmap? Ethereum nodes/L2s with optimistic or zk-proofs are probably the most advanced distributed databases that actually work.

(not talking about "coins" and stuff obviously, another debate)

Re: Jepsen: NATS 2.12.1

#95
post #41

Earlier quoted context omitted.

I always wondered why the fsync has to be lazy. It seems like the fsync's can be bundled up together, and the notification messages held for a few millis while the write completes. Similar to TCP corking. There doesn't need to be one fsync per consensus.

Yes, good call! You can batch up multiple operations into a single call to fsync. You can also tune the number of milliseconds or bytes you're willing to buffer before calling `fsync` to balance latency and throughput. This is how databases like Postgres work by default--see the `commit_delay` option here: https://www.postgresql.org/docs/8.1/runtime-config-wal.html

In some contexts (interrupts) we would call this "coalescing." (I don't work in databases, can't comment about terminology there.)

Re: Jepsen: NATS 2.12.1

#96
post #40

Earlier quoted context omitted.

One of the perks of being distributed, I guess. The kind of failure that a system can tolerate with strict fsync but can't tolerate with lazy fsync (i.e. the software 'confirms' a write to its caller but then crashes) is probably not the kind of failure you'd expect to encounter on a majority of your nodes all at the same time.

It is if they’re in the same physical datacenter. Usually the way this is done is to wait for at least M replicas to fsync, but only require the data to be in memory for the rest. It smooths out the tail latencies, which are quite high for SSDs.

> It smooths out the tail latencies, which are quite high for SSDs.

I'm sorry, tail latencies are high for SSDs? In my experience, the tail latencies are much higher for traditional rotating media (tens of seconds, vs 10s of milliseconds for SSDs).

Re: Jepsen: NATS 2.12.1

#97
post #81

If you are looking for a serverless alternative to JetStream, check out https://s2.dev Pros: unlimited streams with the durability of object storage – JetStream can only do a few K topics Cons: no consumer groups yet, it's on the agenda

Have you tried running Jepsen against it?

We do deterministic simulation testing

https://s2.dev/blog/dst https://s2.dev/blog/linearizability

We have also adopted Antithesis for a more thorough DST environment, and plan to do more with it.

One day we will engage Kyle to Jepsen, too. I'm not sure when though.

Re: Jepsen: NATS 2.12.1

#98
post #29

Earlier quoted context omitted.

Likewise. It took me a moment to realise Jepsen!== Jeppesen

It's named after Carly Rae Jepsen, of 2012 hit single "Call Me Maybe" fame.

I think Aphyr will insist it isn't actually named after Carly Rae for legal reasons, just a striking coincidence.

Re: Jepsen: NATS 2.12.1

#99
post #94

Earlier quoted context omitted.

Like this bug report? The things that have been "disrupted" haven't delivered - Blockchains are still a scam, Food delivery services are worse than before (Restaurants are worse off, the people making the deliveries are worse off), Taxis still needed to go back and vet drivers to ensure that they weren't fiends.

> Blockchains are still a scam Did you actually look at the blockchain nodes implementation as of 2025 and what's in the roadmap? Ethereum nodes/L2s with optimistic or zk-proofs are probably the most advanced distributed databases that actually work. (not talking about "coins" and stuff obviously, another debate)

> Ethereum nodes/L2s with optimistic or zk-proofs are probably the most advanced distributed databases that actually work.

What are you comparing against? Aren't they slower, less convenient, and less available than, say, DynamoDB or Spanner, both of which have been in full-service, reliable operation since 2012?

Re: Jepsen: NATS 2.12.1

#100
post #94

Earlier quoted context omitted.

> Blockchains are still a scam Did you actually look at the blockchain nodes implementation as of 2025 and what's in the roadmap? Ethereum nodes/L2s with optimistic or zk-proofs are probably the most advanced distributed databases that actually work. (not talking about "coins" and stuff obviously, another debate)

> Ethereum nodes/L2s with optimistic or zk-proofs are probably the most advanced distributed databases that actually work. What are you comparing against? Aren't they slower, less convenient, and less available than, say, DynamoDB or Spanner, both of which have been in full-service, reliable operation since 2012?

the big difference is the trust assumption, anyone can join or leave the network of nodes at any time
Post reply on HN