Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Pulsar is an open-source distributed pub-sub messaging system

pulsar.apache.org

Post reply on HN
Print view View original

51–60 of 249 posts

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#51

Post by maximente »

JackRabbitSlim wrote: post #23

Another over engineered Lego block for quicker dev and even less thought on design, upkeep or overhead. Now if you excuse me I need to go take my quad-core, petaflop processing power and multiple gigabytes of RAM to read email from a javascript infested, multi-byte to single byte encoded webpage hosted across half a dozen server instances scattered across the planet. CS is damned, and this is hell.

not really, people legitimately need this for truly big data. you can't reliably processing 7 trillion events per day with a completely C/unix CLI stack.

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#55

Post by Eikon »

buboard wrote: post #53

Why is apache developing all those servers that are only useful to a handful of companies that are rich enough to build them themselves? How about building something that individuals can use, like, i dunno, apache server itself?

If you did bother to read the linked page, you would have understood that it’s a yahoo project handed over to Apache for management like many of Apache’s projects.

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#56

Post by _frkl »

tyri_kai_psomi wrote: post #46
geodel wrote: post #38

Earlier quoted context omitted.

This sound about right. Apart from maybe original Apache HTTP server most of the Apache projects are in Java. Looking at codebase of Pulsar it looks like typical Apache style sprawling Java project with more than thousand directories, many thousand files and more than hundred dependencies. As comparison NATS which is in Go has few hundred files, less than hundred directories and about a dozen or so dependencies.

NATS is an amazing project, I just wanted to take the opportunity to highlight it for those first hearing about it in this comment. It's so brilliantly simple, yet changed the way I design distributed systems. I handle almost anything in regards to the standard messaging guarantees that a Kafka-like system offers at the endpoints now. As a result, systems are much simpler, and diagnosability of bugs or edge cases are…

This sounds interesting, what exactly do you mean by 'endpoint' in this scenario? I looked into a few alternatives before settling for pulsar, and disregarded nats because it didn't seem to support message persistence. I didn't look into it too deeply though, maybe i should have. How do you guarantee no message is lost with NATS?

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#57

Post by buboard »

Eikon wrote: post #55
buboard wrote: post #53

Why is apache developing all those servers that are only useful to a handful of companies that are rich enough to build them themselves? How about building something that individuals can use, like, i dunno, apache server itself?

If you did bother to read the linked page, you would have understood that it’s a yahoo project handed over to Apache for management like many of Apache’s projects.

yeah i m talking more generally about their full list here: https://www.apache.org/

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#58

Post by sz4kerto »

EGreg wrote: post #27
sz4kerto wrote: post #9

Earlier quoted context omitted.

Very different. Pulsar is primarily a Kafka competitor. - it is much more performant than RabbitMQ - it's a commit log as well, not just a pub-sub system, ie. it is a good candidate as the storage backend for event sourcing - it supports geodistributed and tiered storage (eg. some data on NVMe drives, some on a coldline storage) - it's persistent, not in-memory (primarily) .. and so on.

What about ZeroMQ? Why use RabbitMQ and Kafka if you can use ZeroMQ? Meaning, isn’t it far more performant and distributed? Maybe I am missing something here.

> Why use RabbitMQ and Kafka if you can use ZeroMQ?

They are totally different, you're comparing apples with oranges.

ZeroMQ gives you basic, very fast tooling to communicate between distributed processes. ZeroMQ does not provide tooling for e.g. maintaining a strictly ordered, multi-terabyte event log. And so on.

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#59

Post by vips7L »

geodel wrote: post #38
bsaul wrote: post #18

Sidenote question : Are we heading toward a split between apache/java/zookeeper stacks and go/etcd on the other ? I've seen an issue related to that question on pulsar, and this got me investigating the distributed KV part of the stack. It seems by looking at some benchmark that etcd is much more performant than zookeeper, and that to some people, operating two stacks seems like an operation maintenance cost a bit to…

This sound about right. Apart from maybe original Apache HTTP server most of the Apache projects are in Java. Looking at codebase of Pulsar it looks like typical Apache style sprawling Java project with more than thousand directories, many thousand files and more than hundred dependencies. As comparison NATS which is in Go has few hundred files, less than hundred directories and about a dozen or so dependencies.

Its more than one project in the repo.

Re: Apache Pulsar is an open-source distributed pub-sub messaging system

#60

Post by dickjocke »

sz4kerto wrote: post #9
eerrt wrote: post #2

How does this compare to Redis Pub-Sub or RabbitMQ?

Very different. Pulsar is primarily a Kafka competitor. - it is much more performant than RabbitMQ - it's a commit log as well, not just a pub-sub system, ie. it is a good candidate as the storage backend for event sourcing - it supports geodistributed and tiered storage (eg. some data on NVMe drives, some on a coldline storage) - it's persistent, not in-memory (primarily) .. and so on.

Why is one Apache providing competing with another one?
Post reply on HN