Earlier quoted context omitted.
See: https://en.wikipedia.org/wiki/Apache_ZooKeeper https://upload.wikimedia.org/wikipedia/en/thumb/8/81/Apache_...
Those are some impressively sized hands.
Kafka without ZooKeeper
31–40 of 201 posts
Re: Kafka without ZooKeeper
#32Awesome!
Re: Kafka without ZooKeeper
#33Kafka is a pretty cool technology, but for every project that I work on, it's never used because it feels like it's overkill (costly and operation heavy). Maybe I should start looking for bigger projects :D
yeah, really needs a use case that justifies it, I have a particular IoT backend where I made it pluggable between kafka and rabbitmq, ended up just using rabbitmq as it is simpler to work with / manage, and still not really pushing it in terms of performance with thousands of devices.
Re: Kafka without ZooKeeper
#34Re: Kafka without ZooKeeper
#35Re: Kafka without ZooKeeper
#36Oh hell yeah! That's great news, tons of work went into this -- props to the contributors! I will take the opportunity to say that Kafka is kind of painful, with or without ZK. Check out NATS! [0]. It doesn't solve all the same problems, but is so much easier to use (during development especially) and can do a lot of the same things. [0]: https://docs.nats.io/whats_new_20
Using Kafka only for ordered pub-sub is like ordering tap water at French Laundry.
[deleted]
Re: Kafka without ZooKeeper
#37Oh hell yeah! That's great news, tons of work went into this -- props to the contributors! I will take the opportunity to say that Kafka is kind of painful, with or without ZK. Check out NATS! [0]. It doesn't solve all the same problems, but is so much easier to use (during development especially) and can do a lot of the same things. [0]: https://docs.nats.io/whats_new_20
NATS is not a replacement for any use of kafka that I think is actually a good fit.
Re: Kafka without ZooKeeper
#38Oh hell yeah! That's great news, tons of work went into this -- props to the contributors! I will take the opportunity to say that Kafka is kind of painful, with or without ZK. Check out NATS! [0]. It doesn't solve all the same problems, but is so much easier to use (during development especially) and can do a lot of the same things. [0]: https://docs.nats.io/whats_new_20
NATS is not a replacement for any use of kafka that I think is actually a good fit.
NATS by itself doesn’t handle the persistence side at all (by design). It’s at most once delivery, not at least once.
That being said, have you checked out NATS Streaming Server? It’s effectively a first party client for NATS that gives it at least once semantics and persistence, and makes it much more applicable to use cases that are currently on Kafka.
Docs here if you’re curious - https://docs.nats.io/nats-streaming-concepts/intro
Re: Kafka without ZooKeeper
#39Oh hell yeah! That's great news, tons of work went into this -- props to the contributors! I will take the opportunity to say that Kafka is kind of painful, with or without ZK. Check out NATS! [0]. It doesn't solve all the same problems, but is so much easier to use (during development especially) and can do a lot of the same things. [0]: https://docs.nats.io/whats_new_20
I believe NATS Streaming[0] and upcoming Nats Jetstream[1] could be more relevant to those who looking for a Kafka alternative. They offer persistent messages, at-least-once delivery similar to Kafka. [0] https://docs.nats.io/nats-streaming-concepts/intro [1] https://github.com/nats-io/jetstream
[nb: I work for the main company behind NATS]
Jetstream is GA with the 2.2.0 release. Folks who believe in waiting for "not .0" won't have to wait too much longer.
Re: Kafka without ZooKeeper
#40Finally. I assume there must be good reasons beyond "that's what Hadoop has always used" but philosophically, I never understood why introduce yet another network dependency to handle elections. It really adds up to the operational complexity, from having to manage the Zookeeper cluster to having to fight against DNS.