131–140 of 147 posts
Earlier quoted context omitted.
Kafka is not a "db engine". zk is a "db engine" in the same way 'DNS' is a "db engine".
Ah yes, the semantic argument. Fyi - pulsar and etcd do use fsync
Earlier quoted context omitted.
"MongoDB’s default level of write concern was (and remains) acknowledgement by a single node, which means MongoDB may lose data by default." Cassandra doesn't do that, consistency level is fundamental to the documentation and user guide. That is AWFUL. "Curiously, MongoDB omitted any mention of these findings in their MongoDB and Jepsen page. Instead, that page discusses only passing results, makes no mention of read…
The default write concern for the last 2 years has been majority. And single node is a perfectly fine default for most use cases. After all Cassandra's default consistency level is 1.
Earlier quoted context omitted.
Exactly I will never ever try MongoDB because of that. A database that do not fsync should not be called a database.
MongoDB has been doing fsync by default for over a decade now . And those that actually had tried it were aware that every client enabled fsync out of the box. So in fact the entire situation was seriously overblown. But sure let irrational ideology affect your technology decisions. That will work out well.
Not everyone has time to review every single line of code in their tech stacks. Past reputation is important, and your replies here don't seem to be of much help to MongoDB's reputation as far as I can tell.
Earlier quoted context omitted.
As far as read the blog post, I understand that it assumes the scenario that "a replica dies (and loses its log prefix due to no fcyns) and came back instantaneously (before another replica catches up to the leader)". Then, in Kafka, what if the leader dies with power failure and came back instantaneously? i.e.: Let's say there are 3 replicas A(L), B(F), C(F) (L = leader, F = follower) - 1) append a message to A - 2)…
I love this question. Would be great to hear back from Confluent about this. One safety mechanism I can think of is that the replicas will detect the leader is down and trigger leader election themselves. Or that upon restart the leader realized it restarted and triggers leader election in a way that B ends up as the leader. (not sure either is being done) As I think about it more, even if there’s a solution I think…
The situation above may be just one example of data loss, but it seems there could be others when we gamble on hoping servers restart quickly enough, and don’t crash at the same time, etc.
alex here, original author of redpanda is hard to respond to a 6-part blog series content - released all at once - on an HN thread. - what we can deterministically show is data loss on apache kafka with no fsync() [shouldn't be a surprise to anyone] - stay tuned for an update here. - the kafka partition model of one segment per partition could be optimized in both arch - the benefit for all of us, is that all of thes…
Try this on your laptop see global data loss - hint: multi-az is not enough
Regardless of the replication mechanism you must fsync() your data to prevent global data loss in non-Byzantine protocols.
Earlier quoted context omitted.
Yep kafka by default is setup to lose data, many people dont know or dont care it seems…
Exactly because we read the documentation and we use it for things where losing data is acceptable. Just like using HyperLogLog acceptable in many scenarios, using Kafka also acceptable. I am quite baffled how widespread the misuse of technology. Need reliable data storage? Use a database.
Earlier quoted context omitted.
> Because I, as any can anyone, show data loss by running a cluster without fsync and simultaneously pulling the plug on every server. Woah, yeah that's a serious problem. Data loss under that scenario is nothing to sneeze at.
Then enable fsync. I don't really see a way around requiring synchronization to persistent disk if you want persistence cross power outages, right?
Earlier quoted context omitted.
The default write concern for the last 2 years has been majority. And single node is a perfectly fine default for most use cases. After all Cassandra's default consistency level is 1.
Most users of ScyllaDB/Cassandra use quorum or local_quorum
Hence my point there is no difference in defaults between MongoDB and Cassandra.
This was a nice read! There are a few issues on both sides, some that others have mentioned and some that I have not seen yet: For Redpanda: 1. I don't like that they did not include full disk performance, not sure if that was intentional but it feels like it... Seems like and obvious gap in their testing. Perhaps most of their workloads have records time out rather than get pushed out by bytes first, not sure. 2. Th…
This is of course why performance suffers with 50 producers and 288 partitions: not because there is any inherent scale issue in supporting 50 clients (Repanda supports 1000s of clients), but because a 500 MiB/s load spread out among 50 producers and 288 partitions is only ~36 KiB/s per partition-client pair, which is where batching happens. With a linger of 1 ms (the time you'd wait for a batch to form) that's only 36 bytes per linger period so this test is designed to ensure there is no batching at all, to maximize the cost of fsyncs and put Redpanda in a bad light.
A second problem is that most benchmarks, including the one used here, use uniform timings for everything. E.g., when you set the OpenMessaging benchmark to send 1000 messages per second, it schedules a send of one message every 1 millisecond, exactly: i.e., there is no variance in the inter-message timing.
In the real world, message timing is often likely to be much more random, especially when the messages come from external events, like a user click or market event (these are likely to follow a Poisson distribution).
This actually ends up mattering a lot, because message batching will in general be worse under perfect uniformity. E.g., if you have a linger time of 1 ms, a rate of say 900 messages/sec will get no batching (other than forced batching), because each message arrives ~1.1 ms after the last, missing the linger period. If the arrival times were instead random, or especially if they were bursty, you’d get a fair amount of batching just due to randomness, even though the average inter-message time would still be 1.1 ms.
Disclosure: I work at Redpanda.
This was a nice read! There are a few issues on both sides, some that others have mentioned and some that I have not seen yet: For Redpanda: 1. I don't like that they did not include full disk performance, not sure if that was intentional but it feels like it... Seems like and obvious gap in their testing. Perhaps most of their workloads have records time out rather than get pushed out by bytes first, not sure. 2. Th…