101–110 of 147 posts
For Redpanda:
1. I don't like that they did not include full disk performance, not sure if that was intentional but it feels like it... Seems like and obvious gap in their testing. Perhaps most of their workloads have records time out rather than get pushed out by bytes first, not sure.
2. Their benchmark was def selective, sure, but they sell via proof of performance for tested workloads IIUC, no via their posted benchmarks. The posted benchmarks just get them into the proof stage in a sales pipeline.
For Kafka (and Confluent, and this test):
1. Don't turn off fsync for Kafka if you leave it on with Redpanda, that's certainly not a fair test.
Batching should be done on the client side anyway, as most packages already do by default. If you are worried about too many fsyncs degrading performance, batch harder on your clients. It's the better way to batch anyway.
2. If confluent cloud is using java 11, then I don't like that java 17 is used for this either. It's not a fair comparison seeing that most people will want it managed anyways, so it gives unrealistic expectations of what they can get
3. Confluent charges a stupid amount of money
4. The author works for Confluent, so I'm not convinced that this test would have been posted if they saw Redpanda greatly outperform Kafka
With Both:
1. Exactly once delivery is total marketing BS. At least Redpanda mentions you need idempotency, but you get exactly once behavior with full idempotency anyway. What you build should be prepared for this, not the infra you use IMO as all you need is one external system to break this promise for the whole system to lose it
I prefer Redpanda as I find it easier to run, and Redpanda actually cares about their users whether they are paid or not. Confluent wont talk to you unless you have a monthly budget of at least $10k, Redpanda has extremely helpful people in their slack just waiting to talk to you.
Ultimately you don't just buy into the software, you buy into the team backing it, and I'd pick Redpanda easily, knowing that they can actually help me and care without needing to give them $10k.
Earlier quoted context omitted.
I am not entirely sure what is the reason to make Kafka transactional. The original goal was to have a message queue that holds statistical data where the data loss cannot significantly alter the outcome of the analytics performed on the (often incomplete) data. Why are in this argument about fsync and such now? Did something change? If you need reliable data storage do not use Kafka or similar technologies.
Kafka is used widely as a persistent event store, and its development features reflect that. Why would I not just turn on fsync or deploy in a distributed pattern for reliability so I can just continue using it instead of ripping it out, benchmarking something new, teaching the entire org something new, potentially negotiating a new contract, and then executing a huge migration?
Earlier quoted context omitted.
Kafka is used widely as a persistent event store, and its development features reflect that. Why would I not just turn on fsync or deploy in a distributed pattern for reliability so I can just continue using it instead of ripping it out, benchmarking something new, teaching the entire org something new, potentially negotiating a new contract, and then executing a huge migration?
Just like heroin is widely used a recreational drug. We live in a free world and you can use Kafka as a persistent reliable store, even use it transactionally. Instead of reading the marketing claims I like to read what @aphyr has to say about data storage systems. https://aphyr.com/posts/293-call-me-maybe-kafka
Earlier quoted context omitted.
Kafka is used widely as a persistent event store, and its development features reflect that. Why would I not just turn on fsync or deploy in a distributed pattern for reliability so I can just continue using it instead of ripping it out, benchmarking something new, teaching the entire org something new, potentially negotiating a new contract, and then executing a huge migration?
Are you sure performance would be acceptable if you just turned on fsync on every message?
But at a certain point any technology is going to reach the limits of what current hardware and operating system primitives can do.
fsync vs. distributed consensus vs. other tradeoffs w.r.t reliability and consistency are not inherent to "Kafka or similar technologies". It's inherent to anything that runs on a computer in the real world.
Generally unless your scale is mind-bogglingly big, the ROI on tuning what you already have is going to be way way bigger than just ripping it out because you read a benchmarking article.
TLDR: "I work at Confluent, the owners of Kafka, and I have determined through my tests that Redpanda's performance is greatly exaggerated." I don't think we can get a less reliable or trustworthy set of performance tests than when someone's paycheck depends on the outcome of those tests. If Redpanda's performance were found to be better, would he really publish the test results?
Confluent doesn't own Kafka. Apache Kafka is an Apache project, with its own government structure. Some of the project management committee is employed by Confluent, but not all: e.g., the current PMC chair is Mickael Maison, employed by Red Hat. See https://projects.apache.org/committee.html?kafka
TLDR: "I work at Confluent, the owners of Kafka, and I have determined through my tests that Redpanda's performance is greatly exaggerated." I don't think we can get a less reliable or trustworthy set of performance tests than when someone's paycheck depends on the outcome of those tests. If Redpanda's performance were found to be better, would he really publish the test results?
I don't think who does the benchmark, any benchmark, matters as long as they're open about how it was done, what properties were set, ideally why they were set, and what their results were. The big picture goal is to ostensibly be able to reproduce such benchmarks. But I've found through industry that most benchmarks, especially for infrastructure software, are performed by the vendors. The burden for standing up the…
Interestingly that's not necessarily the case in the public cloud. I'm messing around with AWS storage for an upcoming talk. You definitely can saturate storage on AWS, and it's sometimes hard to tell why.
Earlier quoted context omitted.
Well, that just isn't accurate really. Kafka would need simulteanous VM failure across all AZs. That just doesn't happen in the real world often enough to worry about. It has never happened in Confluent Cloud. RP have a similar issue. Single AZ deployments with local NVMe drives. AZ loses power, a majority of brokers could lose all their data. Then there's data corruption. Fsyncs alone don't save you. The next step w…
I vouched for this comment (can we please not, folks?). Sure but many people dont run across AZ bc it costs a ton of money. Fsync alone dont save you but it sure makes it less likely to suffer data loss. > Does a system that has implemented anti-corruption in the storage layer now get to lambast Redpanda, Pulsar, ZooKeeper etc because they didn't implement that? Sure, why not? I think zk doesn’t do fsync too btw
Earlier quoted context omitted.
> There is no call for wanting fsync per message, it is an anti pattern and isn’t done because it isn’t necessary 1. Don't have to do it by message 2. It's used by many distributed db engines, kafka and (i think) zk are the outliers here, not the other way around
Kafka is not a "db engine". zk is a "db engine" in the same way 'DNS' is a "db engine".
[1]: https://dyna53.io
Earlier quoted context omitted.
Confluent doesn't own Kafka. Apache Kafka is an Apache project, with its own government structure. Some of the project management committee is employed by Confluent, but not all: e.g., the current PMC chair is Mickael Maison, employed by Red Hat. See https://projects.apache.org/committee.html?kafka
Kafka PMC is utterly dominated by Confluent or former employees. Everything Kafka does has been and always will be with Confluent's best interest first and foremost. The idea that Kafka isn't completely controlled by Confluent would be disingenuous at best. I don't have anything against Kafka or Confluent, but people should call a spade a spade here when it's blatantly obvious.