Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pulsar vs. Kafka

streamnative.io

Post reply on HN
Print view View original

71–80 of 103 posts

Re: Pulsar vs. Kafka

#71

Post by leafboi »

trengrj wrote: post #15

With Pulsar vs Kafka, I don't see a huge argument between either one functionality wise as they have so much in common (distributed log, Java based, avoid copying memory, use Zookeeper). Because Kafka is more supported and well-known it seems Pulsar needs to be an order of magnitude more performant to capture developer mindshare. I see the same with Spark vs Flink in that similarities outweigh differences. I wonder i…

>it seems Pulsar needs to be an order of magnitude more performant to capture developer mindshare.

Just to add to this, ease of use/setup is also a huge factor. There are technologies I can just spin up with zero knowledge and learn as I go. These are huge factors in adoption especially with Golang and nodejs.

Re: Pulsar vs. Kafka

#72

Post by nikhilsimha »

willvarfar wrote: post #33
doctor_eval wrote: post #25

> Lower end-to-end latency helps enterprises gain business insights faster. They lost me here. I can think of plenty of situations where reduced latency is beneficial, but not many situations where shaving a few milliseconds would make a difference to “business insight”! Although I suppose it is strictly correct, in the tautological sense...

So I've developed an in-house fancy thoroughbred 'real time' data warehouse, a very rare beast indeed, and its awesome. Of course, our business is still running on nightly reports. But the tech is cool! So I wanna say you're wrong, but I've got man-years invested in a system that hasn't been utilized to its full potential yet :( Time will tell. Somewhere, some competitor will be using real-time insights to out-compet…

Nightly to seconds is definitely great. Seconds to milliseconds is what is questionable I think.

Re: Pulsar vs. Kafka

#75

Post by z9e »

abledon wrote: post #73

I still can't believe kafka doesnt have a good open source GUI managing tool

This is great: https://github.com/tchiotludo/akhq, using this currently in our environment. There's also Kafka Manager which has been around for a while.

I still recommend managing Kafka through the CLI tools however.

Re: Pulsar vs. Kafka

#76

Post by throwaway_pdp09 »

nikhilsimha wrote: post #72
willvarfar wrote: post #33

Earlier quoted context omitted.

So I've developed an in-house fancy thoroughbred 'real time' data warehouse, a very rare beast indeed, and its awesome. Of course, our business is still running on nightly reports. But the tech is cool! So I wanna say you're wrong, but I've got man-years invested in a system that hasn't been utilized to its full potential yet :( Time will tell. Somewhere, some competitor will be using real-time insights to out-compet…

Nightly to seconds is definitely great. Seconds to milliseconds is what is questionable I think.

> Nightly to seconds is definitely great

Why? Can a business mobilise in anything less than days? If a report is minutes out of data, is that any loss? Given that some largish proportion of reports are never used, perhaps better management is key.

not disagreeing but efficiency is not just a matter of quickness.

Re: Pulsar vs. Kafka

#77

Post by oweiler »

Tsarbomb wrote: post #46
SmooL wrote: post #40

Is there any sort of 'single node' version of these frameworks? I'm very interested in building event-driven solutions, but I don't need the scale offered by kakfa/pulsar, and I really don't want all the complexity. Is there any reason nobody has made a smaller, less distributed event-centric DB?

You can run kafka on a single node no problem. Give this a try: https://www.digitalocean.com/community/tutorials/how-to-inst...

Single node Kafka still needs Zookeeper, though.

Re: Pulsar vs. Kafka

#78

Post by kevstev »

majidazimi wrote: post #67
toomanybits wrote: post #56

Earlier quoted context omitted.

1. is true, but if you want that data to move to a new node, it still needs to be replicated. Kafka's approach is to use tiered storage (which I believe is close to completion). 2. Kafka can read from a replica node. It's relatively new but it's there.

That's true but still limitation is not fully resolved. In order to increase consumption rate, we need to add replicas. In pulsar Brokers are merely cache nodes over Bookkeeper. Adding more Brokers is trivial in Pulsar.

How in pulsar do they get around the fact that adding a new broker, data needs to be moved over before that broker can start serving data? This seems like a basic law of physics type limitation to me.

Re: Pulsar vs. Kafka

#79

Post by x87678r »

z9e wrote: post #43
x87678r wrote: post #39

Another service that "needs" zookeeper. Has anyone figured out a simple way to manage zookeeper for a small team?

What problems have you had with it?

Its just more complicated than it needs to be. Esp I have a hobby project with tiny usage.

Re: Pulsar vs. Kafka

#80

Post by lymitshn »

SmooL wrote: post #40

Is there any sort of 'single node' version of these frameworks? I'm very interested in building event-driven solutions, but I don't need the scale offered by kakfa/pulsar, and I really don't want all the complexity. Is there any reason nobody has made a smaller, less distributed event-centric DB?

nats streaming worth checking out, or if you dont care about the streaming part nats/zeromq should be simple enough (simple pub/sub infra tools).
Post reply on HN