Live data from Hacker News

ActiveMQ: Not ready for prime time

goodstuff.im

61–70 of 93 posts

Re: ActiveMQ: Not ready for prime time

#61
There is a lot of buzz about open source queue systems. I have tried for example the RabbitMQ with very bad results. I'll rather explain... one interesting concept about queues is contention and this is where RabbitMQ and others are behind. The strange thing is that contention is an old concept from the old mainframes.

For example the last time I checked you can't block a produced in RabbitMQ based on the number of messages on the queue.

Re: ActiveMQ: Not ready for prime time

#62
post #13

How does activemq compare to zeromq?

zeromq does not persist messages so you can lose them and need to adjust your infrastructure around this. If you are used to network programming that should be fine, just build persistence in to your protocol where you need it, ie reply after you have committed to persistent storage.

It persists messages in one direction. Imagine you have a publish subscribe situation where you have a few apps subscribed to a publisher. If one of those apps goes down, the publisher will make a note and store the messages until it comes back. When the app comes back, it will get all the messages it missed. It's really a wonderful thing if you were going to implement peer-to-peer pub/sub yourself.

It's also nice in applications like Mongrel2. Make an http request, in your browser, start up your web server, and the request completes! Great for development, where sometimes you switch to the browser faster than your app can restart after a change. With Mongrel2 (and thanks to 0MQ), you don't even notice the race condition.

Re: ActiveMQ: Not ready for prime time

#63
post #8

sorta reminds me of the time I built something in 2001 which was a reactive system that was sort of a web crawler but also a bit more... seems demented looking back, but I used qmail as a message queue system for it.

i've considered xmpp as a queue, and hear some do use it!

Yes - I've been seriously considering XMPP for this purpose. I haven't seen any serious reason not to consider it other than some people's dislike of IM protocols.

Re: ActiveMQ: Not ready for prime time

#64

I used ActiveMQ in a large production environment for two years. It's by a large margin the worst piece of software I've used professionally. You wouldn't believe the number of very serious problems we had with it (including some mentioned here, like negative queue lengths, but mostly broker crashes, missing or wrong documentation, outright broken features, serious threading issues, and poor performance). If anyone's…

This is refreshing to read. I worked with ActiveMQ for about 2 years, and had nothing but problems with it. We probably lost thousands of dollars in sales because of lockups, corruptions, etc. I always assumed that we were "just doing something wrong" (which we probably were in many cases), but we were basically using out-of-the-box configurations and Spring for the JMS support. I'm glad to hear that I'm not the only one that has had problems with it. I assumed it should be pretty stable since it's an Apache project, but I have to agree that the source code is almost incomprehensible.

Re: ActiveMQ: Not ready for prime time

#65
post #9

The thing I've found about message queues is that it's pretty unimportant what platform they're based on. I used to be tempted to go with ActiveMQ "because it runs on the JVM", but that turns out to be pretty irrelevant. No matter what your queue provider, you're most likely just using TCP sockets anyway, so don't think you need to be using Erlang just to take advantage of RabbitMQ. Yes, this means the MQ can basical…

What I've seen with RabbitMQ is that the memory usage can grow and grow. If it runs out of either committed RAM or address space, it crashes. I suspect this may be a feature of the open source Erlang VM that's improved in the pay-for-support version. Even though RabbitMQ pays the performance cost of bouncing everything off the disk, it's not quite recoverable from that type of crash. I'm sure my information is old, p…

What I've seen with RabbitMQ is that the memory usage can grow and grow

Prior to 2.0, the contents of Rabbit's queues were always in RAM. Even durable queues with persistent messages always had their contents mirrored in RAM. As of 2.0, Rabbit can store queue contents on disk or in RAM, so as it faces memory pressure, it will send data to disk. If the disk is too slow for it to relieve its pressure, it will block connections that have sent data in the past until it can free up enough memory to continue.

Last I checked, Rabbit does by default keep the associative map between queues and message in RAM, so its memory usage isn't technically bounded (it's a around a dozen bytes per message, IIRC). If that's a problem, the toke plugin uses tokyo cabinet to store the associations, which I believe allows Rabbit to have entirely bounded memory use. I also haven't checked on that for a while, so it may be that toke is installed by default now.

Re: ActiveMQ: Not ready for prime time

#66

I used ActiveMQ in a large production environment for two years. It's by a large margin the worst piece of software I've used professionally. You wouldn't believe the number of very serious problems we had with it (including some mentioned here, like negative queue lengths, but mostly broker crashes, missing or wrong documentation, outright broken features, serious threading issues, and poor performance). If anyone's…

I had this exact experience about four years ago. Spent way too long dealing with their buggy code, wrote our own in a few weeks and everything was much easier from then on.

Re: ActiveMQ: Not ready for prime time

#67

It's absolutely incredible that it can take so much effort just to make it run. ActiveMQ seems shiny and interesting on the outside, but the prolific number of war stories took it off my list. In brilliant contrast, RabbitMQ is it-just-works software. It's a piece of infrastructure that works so well it practically just fades into the background. I couldn't imagine having to worry about a service in which the chief g…

The Reddit guys think differently: "[...] (rabbitmq) died, which added about an hour to the downtime. It dies like this pretty often at 2am or at other especially bad times. Usually it doesn't cause any data-loss, just sleep-loss (its queues are persisted and the apps just build up their own queues until it comes back up), but in this case it decided to crash in a way that corrupted its database of persisted queues beyond repair. rabbitmq accounts for the only unrecoverable data-loss incurred, which was about 400 votes. [...] Coincidentally, rabbitmq crashed twice more that day and a few more times into the weekend. [...] Things have improved thus far, but replacing rabbitmq is at the top end of our extremely long list of things to do."

Re: ActiveMQ: Not ready for prime time

#68
post #10

> It's not correctly configured out of the box. You mean there's a correct configuration? Also, Tomcat, Jetty, PostgreSQL, and Ngnix don't "install correctly" out of the box as described here. Tomcat clusters set themselves up? Postgres? Hell, I think default shared buffers allocation is 32 MB... > Anything that streams bytes I'm not sure what the author is trying to do. Doesn't sound like queuing to me... The docume…

Our Tomcats definetly didn't check for file handles, or run out of the box. The same goes for our NGINX. Can't talk about PostgreSQL, but MySQL certainly isn't delivered in a high volume configuration either.

Re: ActiveMQ: Not ready for prime time

#69
I've now been involved in two projects where I would have needed a queue with following characteristics: - simple pub-sub, queue-like: m producers, n consumers - decent performance (order of hundreds of requests per second) - high availability for message persistence (I don't want to loose messages) - no strict FIFO needed, just some kind of lesser fairness (newer items should not block older ones from passing through)

ActiveMQ with active-backup setup over shared disk mount is the current choice, and the start-up is really slow if the queue has a lot of data.

RabbitMQ does not persist messages in HA fashion, so I've ignored it so far. Maybe HornetQ needs some attention.

I see a lot of flexibility and feature-richness in the queue landscape and it perplexes me that getting this simple combination of basics right is so difficult.

Re: ActiveMQ: Not ready for prime time

#70

I used ActiveMQ in a large production environment for two years. It's by a large margin the worst piece of software I've used professionally. You wouldn't believe the number of very serious problems we had with it (including some mentioned here, like negative queue lengths, but mostly broker crashes, missing or wrong documentation, outright broken features, serious threading issues, and poor performance). If anyone's…

We've had nothing but problems with ActiveMQ and we've been using it for 4-5 year now. We've been slow on making the decision to move to a different open source implementation because it is difficult to test this kind of software. I mean, ActiveMQ appears to work fine in tests, but in production, lock-ups and storage leaks happen on monthy basis. We need simple functionality, peristent queues in their simplest form. HornetQ seems like a good candidate. I'd appreciate if anyone could share their experience with it.
Post reply on HN