Viewing profile — lmsp

This is what Apache Pulsar ( https://pulsar.incubator.apache.org/ ) already provides - infinite streaming storage, with simple/flexible messaging streaming API and kafka compatible…

I do think it is technically possible base on my understanding of pulsar - pulsar provides an failover subscription mode, which seems to be the equivalent of partition rebalancing …

Kafka streams is great. But it is a processing library. It isn’t an apple-to-apple comparison to a messaging system like Pulsar. First there are already many mature stream processi…

I think it is beyond partition rebalancing. There is a fact that people didn't realize of making message broker `stateless`. It is actually much better on reacting to failures or s…

another point added to 'rebalancing' -- when kafka rebalances the partitions, it has to copy all the data for the partitions that are moved around. it might not be a big problem wh…

actually, pulsar just supports kafka 0.10 api in the recent apache release. it might be worth checking it out as a drop in replacement. https://github.com/apache/incubator-pulsar/r…

another side note, pulsar supports kafka api since 1.20.0 - https://github.com/apache/incubator-pulsar/releases/tag/v1.2...

you might want to checkout https://pulsar.apache.org/ a durable low latency pub/sub system. it also has a kafka api client.