Kafka vs. Redpanda performance – do the claims add up?
jack-vanlightly.com
Kafka vs. Redpanda performance – do the claims add up?
1–10 of 147 posts
Re: Kafka vs. Redpanda performance – do the claims add up?
#2"I hope you come away with a new appreciation that trade-offs exist, there is no free lunch despite the implementation language or algorithms used. Optimizations exist, but you can’t optimize for everything. In distributed systems you won’t find companies or projects that state that they optimized for CAP in the CAP theorem. Equally, we can’t optimize for high throughput, low latency, low cost, high availability and high durability all at the same time. As system builders we have to choose our trade-offs, that single silver-bullet architecture is still out there, we haven’t found it yet."
Re: Kafka vs. Redpanda performance – do the claims add up?
#3I'd like to see these on OpenJDK 11, since that's what Confluent is running on and the author makes a point of switching to 17 even though he works for Confluent.
In either case, Confluent Platform is ridiculously expensive and approached the costs (licensing alone) for our entire cloud spend. I'd love to see more run-on-k8s alternatives to CFK.
Re: Kafka vs. Redpanda performance – do the claims add up?
#4> Redpanda end-to-end latency of their 1 GB/s benchmark increased by a large amount once the brokers reached their data retention limit and started deleting segment files. Current benchmarks are based on empty drive performance.
This seems really disingenuous to use empty drive performance, since anyone who cares about performance is going to be caring about continuous use.
Re: Kafka vs. Redpanda performance – do the claims add up?
#5I'd like to see these on OpenJDK 11, since that's what Confluent is running on and the author makes a point of switching to 17 even though he works for Confluent. In either case, Confluent Platform is ridiculously expensive and approached the costs (licensing alone) for our entire cloud spend. I'd love to see more run-on-k8s alternatives to CFK.
I agree with Confluent pricing, we had the same experience. We switched to pub/sub and Azure Event hubs.
I don't even understand why Confluent should price their offering so high. ITs not like Real time is an exclusive service that other platforms don't have.
Re: Kafka vs. Redpanda performance – do the claims add up?
#6I'd like to see these on OpenJDK 11, since that's what Confluent is running on and the author makes a point of switching to 17 even though he works for Confluent. In either case, Confluent Platform is ridiculously expensive and approached the costs (licensing alone) for our entire cloud spend. I'd love to see more run-on-k8s alternatives to CFK.
Have you checked Strimzi for Kafka on k8s? it’s super good
Re: Kafka vs. Redpanda performance – do the claims add up?
#7I'd like to see these on OpenJDK 11, since that's what Confluent is running on and the author makes a point of switching to 17 even though he works for Confluent. In either case, Confluent Platform is ridiculously expensive and approached the costs (licensing alone) for our entire cloud spend. I'd love to see more run-on-k8s alternatives to CFK.
I agree with Confluent pricing, we had the same experience. We switched to pub/sub and Azure Event hubs. I don't even understand why Confluent should price their offering so high. ITs not like Real time is an exclusive service that other platforms don't have.
Because they have stock they need to pump.
I've found talking to Confluent about anything is a complete waste of time unless it's a very specific technical issue. They're always pushing their cloud as the solution, and it's very aggressive.
Re: Kafka vs. Redpanda performance – do the claims add up?
#8I'd like to see these on OpenJDK 11, since that's what Confluent is running on and the author makes a point of switching to 17 even though he works for Confluent. In either case, Confluent Platform is ridiculously expensive and approached the costs (licensing alone) for our entire cloud spend. I'd love to see more run-on-k8s alternatives to CFK.
Have you checked Strimzi for Kafka on k8s? it’s super good
Strimzi is really great, creating a Debezium Change Data Capture system and seeing all topics and users as Kubernetes CRDs is just ordered and magic.
The only downside is that Redpanda isn't yet supported in Strimzi, but when I met them at KubeCon last month they mentioned the possibility it might be supported in future :)
Re: Kafka vs. Redpanda performance – do the claims add up?
#9EDIT: Thank you for clarification. It is a fair 3 node vs 3 node benchmark.
Does this benchmark compare both 3 node Kafka against 3 node Redpanda cluster? It's unclear.
Re: Kafka vs. Redpanda performance – do the claims add up?
#10I'd like to see these on OpenJDK 11, since that's what Confluent is running on and the author makes a point of switching to 17 even though he works for Confluent. In either case, Confluent Platform is ridiculously expensive and approached the costs (licensing alone) for our entire cloud spend. I'd love to see more run-on-k8s alternatives to CFK.
I really dislike the way Confluent has treated the Kafka ecosystem. It feels like they went out of their way to make OSS Kafka kludgy and then priced their enterprise offering completely out of the reach of anyone but Fortune 500.
I have been using Pulsar for new projects not because of performance or anything but because all the features you expect to be built-in are. Georeplication, shared-subscription w/selective ACK, schema registry etc.
Also it's wildly more pluggable, the authn/authz plugin infrastruction in particular is great. I was even able to write a custom Pulsar segment compactor to do GDPR deletions without giving up offloaded segment longevity.
The segment offload is actually huge especially because tools like Secor for Kafka are dead now and you are stuck on the Kafka Connect ecosystem which personally I really find distasteful.