Live data from Hacker News

Jepsen: NATS 2.12.1

jepsen.io

21–30 of 176 posts

Re: Jepsen: NATS 2.12.1

#21
post #10

NATS is a fantastic piece of software. But doc’s unpractical and half backed. That’s a shame to be required to retro engineer the software from GitHub to know the auth schemes.

[flagged]

Interested to know if you found these issues yourself or from a source. Is Kafka any more robust?

Re: Jepsen: NATS 2.12.1

#22
post #14

> By default, NATS only flushes data to disk every two minutes, but acknowledges operations immediately. This approach can lead to the loss of committed writes when several nodes experience a power failure, kernel crash, or hardware fault concurrently—or in rapid succession (#7564). I am getting strong early MongoDB vibes. "Look how fast it is, it's web-scale!". Well, if you don't fsync, you'll go fast, but you'll go…

NATS data is ephemeral in many cases anyhow, so it makes a bit more sense here. If you wanted something fully durable with a stronger persistence story you'd probably use Kafka anyhow.

Re: Jepsen: NATS 2.12.1

#24
post #14

> By default, NATS only flushes data to disk every two minutes, but acknowledges operations immediately. This approach can lead to the loss of committed writes when several nodes experience a power failure, kernel crash, or hardware fault concurrently—or in rapid succession (#7564). I am getting strong early MongoDB vibes. "Look how fast it is, it's web-scale!". Well, if you don't fsync, you'll go fast, but you'll go…

Not flushing on every write is a very common tradeoff of speed over durability. Filesystems, databases, all kinds of systems do this. They have some hacks to prevent it from corrupting the entire dataset, but lost writes are accepted. You can often prevent this by enabling an option or tuning a parameter.

> I wouldn't trust a product that doesn't default to safest options

This would make most products suck, and require a crap-ton of manual fixes and tuning that most people would hate, if they even got the tuning right. You have to actually do some work yourself to make a system behave the way you require.

For example, Postgres' isolation level is weak by default, leading to race conditions. You have to explicitly enable serialization to avoid it, which is a performance penalty. (https://martin.kleppmann.com/2014/11/25/hermitage-testing-th...)

Re: Jepsen: NATS 2.12.1

#25

NATS is a fantastic piece of software. But doc’s unpractical and half backed. That’s a shame to be required to retro engineer the software from GitHub to know the auth schemes.

[deleted]

Re: Jepsen: NATS 2.12.1

#26
post #10

NATS is a fantastic piece of software. But doc’s unpractical and half backed. That’s a shame to be required to retro engineer the software from GitHub to know the auth schemes.

[flagged]

NATS was originally made for simple, fast, ephemeral messaging.

The persistence stuff is kinda new and it's not a surprise that there are limitations and bugs.

You should see this report as a good thing, as it will add pressure for improvements.

Re: Jepsen: NATS 2.12.1

#27
post #20
post #16

Earlier quoted context omitted.

"PostgreSQL used fsync incorrectly for 20 years" https://archive.fosdem.org/2019/schedule/event/postgresql_fs... It did not prevent people from using it. You won't find a database that has the perfect durability, ease of use, performance ect.. It's all about tradeoffs.

Realistically speaking, postgresql wasn’t handling a failed call to fsync, which is wrong: but materially different from a bad design or errors in logic stemming from many areas. Postgresql was able to fix their bug in 3 lines of code, how many for the parent system? I understand your core thesis (sometimes durability guarantees aren’t as needed as we think) but in postgresql’s case, the edge was incredibly thin. It…

NATS allows you to fsync every calls, it's not just the default value.

Re: Jepsen: NATS 2.12.1

#28
post #14

> By default, NATS only flushes data to disk every two minutes, but acknowledges operations immediately. This approach can lead to the loss of committed writes when several nodes experience a power failure, kernel crash, or hardware fault concurrently—or in rapid succession (#7564). I am getting strong early MongoDB vibes. "Look how fast it is, it's web-scale!". Well, if you don't fsync, you'll go fast, but you'll go…

NATS data is ephemeral in many cases anyhow, so it makes a bit more sense here. If you wanted something fully durable with a stronger persistence story you'd probably use Kafka anyhow.

Core nats is ephemeral. Jetstream is meant to be persisted, and presented as a replacement for kafka

Re: Jepsen: NATS 2.12.1

#30
post #5

> 3.4 Lazy fsync by Default Why? Why do some databases do that? To have better performance in benchmarks? It’s not like that it’s ok to do that if you have a better default or at least write a lot about it. But especially when you run stuff in a small cluster you get bitten by stuff like that.

Massively improves benchmark performance. Like 5-10x
Post reply on HN