Kafka on Kubernetes: What could go wrong?
redpanda.com
Kafka on Kubernetes: What could go wrong?
1–10 of 29 posts
Re: Kafka on Kubernetes: What could go wrong?
#2TL;DR This is a plug for Red panda.
If you're going to look at Kafka inside of Kubernetes at all just look at running Apache Pulsar in K8s.
One of the main arguments in this article is worry about exposing the broker but in Pulsar there is a proxy for this and the helm chart supports ingress public endpoint by just enabling a flag.
Re: Kafka on Kubernetes: What could go wrong?
#3TL;DR This is a plug for Red panda. If you're going to look at Kafka inside of Kubernetes at all just look at running Apache Pulsar in K8s. One of the main arguments in this article is worry about exposing the broker but in Pulsar there is a proxy for this and the helm chart supports ingress public endpoint by just enabling a flag.
Is there somewhere a real list of pros of cons of pulsar vs kafka ( or redpanda) , from people having real life experience of the two ( and not just reading the documentation) ?
Re: Kafka on Kubernetes: What could go wrong?
#4Doesn't Strimzi make this completely painless?
Re: Kafka on Kubernetes: What could go wrong?
#5We run Kafka with kubernetes, but its so locked down/customized we might as well run it on bare metal. We started out on rook network storage, but after multiple fs corruption incidents, we now run every broker with local storage. Kube still gives us the benefit of re-using our deployment software and many monitoring conventions, so we do get some benefit from kubernetes.
The only reason we did not lose data on the rook setup, is we run 5 brokers with replication factor 4, with 2 in sync.
Re: Kafka on Kubernetes: What could go wrong?
#6TL;DR This is a plug for Red panda. If you're going to look at Kafka inside of Kubernetes at all just look at running Apache Pulsar in K8s. One of the main arguments in this article is worry about exposing the broker but in Pulsar there is a proxy for this and the helm chart supports ingress public endpoint by just enabling a flag.
Is there somewhere a real list of pros of cons of pulsar vs kafka ( or redpanda) , from people having real life experience of the two ( and not just reading the documentation) ?
Btw, speaking of pros and cons, it would be really nice to have a feature comparison website for different types of technologies (e.g. relational databases, messaging systems, etc). For some, there are Wikipedia "Comparison of X" pages, and to some degree you can extract and discover via Stackshare, but Wikipedia is far from complete and up to date, and Stackshare mostly just shows you who uses what and describes technologies via marketing buzzwords rather than somewhat "normalized" descriptions.
Re: Kafka on Kubernetes: What could go wrong?
#7Doesn't Strimzi make this completely painless?
[deleted]
Re: Kafka on Kubernetes: What could go wrong?
#8TL;DR This is a plug for Red panda. If you're going to look at Kafka inside of Kubernetes at all just look at running Apache Pulsar in K8s. One of the main arguments in this article is worry about exposing the broker but in Pulsar there is a proxy for this and the helm chart supports ingress public endpoint by just enabling a flag.
[deleted]
Re: Kafka on Kubernetes: What could go wrong?
#9This is an ad for Redpanda and it's BSL, please stay away
Re: Kafka on Kubernetes: What could go wrong?
#10I would recommend looking at Strmzi for running Kafka on K8S. It works well and addresses all the issues mentioned in this piece. There's no reason to switch to RedPanda because you're running Kubernetes.