Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

x-aeon.com

Post reply on HN
Print view View original

31–40 of 46 posts

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#31

Post by jdcryans »

maybenot wrote: post #17
noselasd wrote: post #13

Please keep in mind: - All these products have a lot of knobs to set. Performance depends on these knobs. It depends on whether you do batch sends(which some can do), whether you do transactions, whether you chose at-least-once, at-most-once etc. delivery, queue prefetch sizes, I/O worker size, and numerous other settings. It'd also be interresting to see qpid-cpp in there as well, not just the Java version.

This is a test of the default configurations of various brokers, NOT the brokers themselves. Apart from declaring the testing queues in some brokers’ configuration and the persistence settings, all brokers were running with their default configuration out of the box (no tuning made). He also casually mentions that ZeroMQ is using an in-memory configuration. A home-made ZeroMQ 2.2.0 broker, working in memory only (no…

My thoughts exactly, this is useless. Benchmarking is something everyone can do, so people do it, but it doesn't provide any real value unless you put an insane amount of time into it, and even then you'll get a lot of hate mail :)

For example, look at the "Yahoo! Cloud System Benchmark" AKA YCSB paper. Those researchers spent a lot of time designing the benchmarks and properly configuring each database that they were testing. They even dedicated a researcher per database to sit down with the developers to review the configurations and the test runs. I was part of this process for the Apache HBase as a dev. In the end, everyone was still critical of the results once they saw the graphs comparing the DBs.

But, I'm still glad they did it as I use YCSB as one of my tools to benchmark HBase.

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#32

Post by sneak »

FooBarWidget wrote: post #28
sneak wrote: post #15

Earlier quoted context omitted.

ZeroMQ is a messaging library only. There is nothing stopping you from using ZeroMQ to build a daemon that implements a persistent queue— in fact, I have, several times. It's very simple.

And then you've made your own broker. At that point why not use an existing, more mature broker?

The zero in ZeroMQ is because brokerlessness is a strength in some aspects.

A broker can be an SPOF, for instance. 0MQ makes it easy to do shared-nothing.

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#33

Post by rabbitmq »

Sidebar --- anyone interested in running tests with RabbitMQ should check out these posts as a starting point.

http://www.rabbitmq.com/blog/2012/04/17/rabbitmq-performance...

http://www.rabbitmq.com/blog/2012/04/25/rabbitmq-performance...

Please note that:

1. the results are from last year, and so not 100% representative of RabbitMQ 3.x which was released after these blogs were posted

2. what actually matters with message queue performance is stability and scalability over time .. testing this is extremely hard without making over-specific assumptions

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#34

Post by rabbitmq »

mootothemax wrote: post #3

Interesting results! In my testing, I came to a different conclusion. Background: PHP and STOMP, with loads of several million messages per hour, ranging in size from 2-20Kb. Multiple processes enqueuing, and many more dequeuing. Tests performed about this time last year. Out of ActiveMQ, RabbitMQ and Apollo: ActiveMQ crashed constantly under load. RabbitMQ could not enqueue/dequeue fast enough. Apollo blew them all…

"RabbitMQ could not enqueue/dequeue fast enough"? I think the STOMP plugin was still pre-production a year ago. You may want to try using RabbitMQ 3.x..

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#35

Post by DomBlack »

mootothemax wrote: post #6
DomBlack wrote: post #5

I'm interested to see ZMQ being compared to the traditional brokers as I've been considering swapping from ActiveMQ. Although not surprised at it's results. It very much re-enforces the ideal of keeping software simple and barebones, rather than bloating it with stuff most people never use. Mainly because I'm finding the latency from ActiveMQ is starting to affect my overall system latency. RabbitMQ was the next brok…

Background; I'm currently running a system which at peak runs with about 15 million messages per hour using ActiveMQ, with several producers and consumers on the same topic. Apart from speed, not had any issues with it. ActiveMQ or ActiveMQ Apollo? Definitely give Apollo and try if you haven't, it's incredibly easy to drop it and requires very little configuration. I'm processing similar peak levels with Apollo, and…

ActiveMQ or ActiveMQ Apollo?

ActiveMQ, haven't looked too much into Apollo since almost it's original announcement. How is it from a stability standpoint?

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#36

Post by DomBlack »

FooBarWidget wrote: post #9
DomBlack wrote: post #5

I'm interested to see ZMQ being compared to the traditional brokers as I've been considering swapping from ActiveMQ. Although not surprised at it's results. It very much re-enforces the ideal of keeping software simple and barebones, rather than bloating it with stuff most people never use. Mainly because I'm finding the latency from ActiveMQ is starting to affect my overall system latency. RabbitMQ was the next brok…

I would not use ZeroMQ for anything but the most non-critical queues. ZeroMQ is not persistent: if your server crashes you lose all your queue contents. My current favorite is RabbitMQ. It has improved steadily over the years, performs pretty well and very easy to setup.

ZeroMQ is not persistent: if your server crashes you lose all your queue contents.

With my requirements if I haven't processed the message quick enough then I don't care if it gets lost, as it would already be outdated data.

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#38

Post by mootothemax »

rabbitmq wrote: post #34
mootothemax wrote: post #3

Interesting results! In my testing, I came to a different conclusion. Background: PHP and STOMP, with loads of several million messages per hour, ranging in size from 2-20Kb. Multiple processes enqueuing, and many more dequeuing. Tests performed about this time last year. Out of ActiveMQ, RabbitMQ and Apollo: ActiveMQ crashed constantly under load. RabbitMQ could not enqueue/dequeue fast enough. Apollo blew them all…

"RabbitMQ could not enqueue/dequeue fast enough"? I think the STOMP plugin was still pre-production a year ago. You may want to try using RabbitMQ 3.x..

I think the STOMP plugin was still pre-production a year ago. You may want to try using RabbitMQ 3.x.

Cool, thanks, I'll take a look and see how we go :)

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#39

Post by mootothemax »

DomBlack wrote: post #35
mootothemax wrote: post #6

Earlier quoted context omitted.

Background; I'm currently running a system which at peak runs with about 15 million messages per hour using ActiveMQ, with several producers and consumers on the same topic. Apart from speed, not had any issues with it. ActiveMQ or ActiveMQ Apollo? Definitely give Apollo and try if you haven't, it's incredibly easy to drop it and requires very little configuration. I'm processing similar peak levels with Apollo, and…

ActiveMQ or ActiveMQ Apollo? ActiveMQ, haven't looked too much into Apollo since almost it's original announcement. How is it from a stability standpoint?

How is it from a stability standpoint?

The only issues I encountered were been due to not assigning enough memory to the JVM. Other than that it's done the job admirably.

Re: A quick message queue benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo

#40

Post by matticakes »

knodi wrote: post #22

Anyone here use NSQ in production ( https://github.com/bitly/nsq ). If so what are your thoughts in comparison to the article?

there are a few (growing) number of production installations of NSQ...

our (bitly's) cluster spans a a few datacenters and hits peaks of 80k messages/second.

I can answer any questions you have (one of the authors)

Post reply on HN