Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An introduction to RabbitMQ

erlang-solutions.com

Post reply on HN
Print view View original

211–220 of 263 posts

Re: An introduction to RabbitMQ

#211

Post by jonathanoliver »

heinrichhartman wrote: post #44
dirtydroog wrote: post #26

We use ZeroMQ a bit. It's been pretty much flawless as far as I can see but I get the impression that it's becoming obsolete. Is RabbitMQ a viable replacement?

You might find it interesting to note, that Peter Hintjens, was one of the core authors of the AMQP 0-9-1 Specification [1], that RabbitMQ is implementing. ZeroMQ was born out of a frustration with complex routing patterns and the need for a broker-less architecture for maximal performance message delivery. [1] https://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf

He and Martin Sustrik both created ZeroMQ. Then after that, they saw some of the limits of ZMQ and created nanomsg. It's excited to see what cool stuff they were working on. It's a little hard to see ZeroMQ become abandonware from them. That said, the community is solid and supportive around ZeroMQ which actually I would say is the best part. In other words, you can tell if a project has staying power when the original creator no longer has to be there to maintain it.

Re: An introduction to RabbitMQ

#212

Post by kabacha »

formerlurker2 wrote: post #180
kabacha wrote: post #177

I'm really surprised to see so much positivity about rabbitmq here when it's probably the most sweared-at software in the space. Let me share my anecdote. In my last work place I got onboarded on rabbitmq and it was such a painful software to work with and almost impossible to set up locally that I silently sneaked in simple redis list as queue alternative for my dev environment. The whole rabbitmq and it's pika libr…

I'd be interested to know what the 3 lines of python were and also more details about the redis server you deployed to replace RabbitMQ during the outage.

I'm using a bit of a hyperbole but using redis list as a task queue is a simple as:

    while True:
        msg = r.lpop(key)
        try:
            result = do_something(msg)
        except Exception as e:
            log.error(f'failure for msg "{msg}" got {e} back to queue {key}')
            r.rpush(key, msg)
            continue
Pop a member from a list, if failed plop it back to the end. It's simple, explicit and just works™

Re: An introduction to RabbitMQ

#213

Post by tmarice »

adamkf wrote: post #191

I've had truly terrible experiences with RabbitMQ. I believe that it should not be used in any application where message loss is not acceptable. Its two big problems are that it cannot tolerate network partitions (reason enough to never use it in production systems, see https://twitter.com/antifuchs/status/735628465924243456 ), and it provides no backpressure to producers when it starts running out of memory. In my l…

Rabbitmq blocks producers when it hits memory high watermark (default 40% of available RAM) - https://www.rabbitmq.com/memory.html

Re: An introduction to RabbitMQ

#214

Post by fake-name »

I've been using rabbitmq heavily for a fairly large hobby project (20-100 messages/sec) for a few years now. I'm generally happy with it, but there are a number of caveats I've learnt.

1. If you have large messages and use keepalives (and you'll need keepalives), you need to write your own message fragmentation.

2. There are no python libs that just work. I'm currently using a vendored version of amqpstorm with a bunch of hacks to handle wedged connections. I have some AMQP connections that are intercontinental, and I've been able to wedge literally every other AMQP library.

3. If you have a single open connection, it will get stuck from time-to-time. With a bunch of both in-band and out-of-band keepalives, I've got it to the point where I don't have things permanently block, but you should expect things getting stuff for ~2x your heartbeat time periodically. This doesn't seem to result in message loss. I've dealt with this by just running LOTS of concurrent connections, and aggregating them client side. This has worked fine.

4. In general, exactly-once delivery isn't a thing. You should design either for at-most-once, or at-least-once delivery modes exclusively. Idempotency is your friend.

5. The tooling /around/ the rabbitmq server is a dumpsterfire.

Basically, I feel like the core server is super durable (note: I'm not running a cluster, so this doesn't generalize to multi-instance cases), but the management stuff is god-awful. The main management CLI tool actually calls the HTTP interface, which is kind of ridiculous. I've occationally run into a situation where I wound up with leaking temporary exchanges, and just flushing bogus exchanges is super annoying.

I don't think there's any other options that can do what rabbitmq does for my use-case, but it's had quite the learning curve.

Re: An introduction to RabbitMQ

#215

Post by mister_hn »

kabacha wrote: post #177

I'm really surprised to see so much positivity about rabbitmq here when it's probably the most sweared-at software in the space. Let me share my anecdote. In my last work place I got onboarded on rabbitmq and it was such a painful software to work with and almost impossible to set up locally that I silently sneaked in simple redis list as queue alternative for my dev environment. The whole rabbitmq and it's pika libr…

With docker it takes 5 seconds to set a rabbitmq instance locally.

Re: An introduction to RabbitMQ

#216

Post by hrpnk »

theptip wrote: post #139
dkersten wrote: post #72

Earlier quoted context omitted.

Was it easy to setup in terms of reliability and failover? Given what you and larrik are saying, I think I need to give it a trial run, but its a project with a tiny team, so I want to be sure it won't be the cause of sleepless nights when things go wrong. It sounds like RabbitMQ is quite solid and shouldn't be the cause for concern, which is promising! Is there anything I should keep in mind for running it in produc…

In my experience RMQ is solid enough that for many use-cases it's reasonable to run it without a standby (especially if you're on Kubernetes where you'll get a replacement instance created automatically if your active instance fails). A common use-case is for async tasks (Celery) that can tolerate a few minutes of downtime. If you're running a fully evented architecture then this might not apply - though if you're no…

Which Kubernetes operator for RabbitMQ are you using?

Re: An introduction to RabbitMQ

#217

Post by rexarex »

I have never had a good experience with RabbitMQ wherever I have worked. Often it was buggy and unreliable. It’s almost always been some thing shoehorned into a service, but failed to gain widespread adoption with future services. Furthermore, it’s usually some hot potato no one even wants to deal with. We have written some code around it to make it more reliable. You quickly figure out why there seems to be so many half baked implementations of it wherever you go work.

It’s basically caught between being too bloated and complex for use with smaller systems (as some commenters have poked at people for not being the ‘right’ kind of person to be running it)

While at the same time, it’s not robust and reliable enough to use in prime time.

What’s left is this enticing and sexy sounding message broker called RabbitMQ that actually just sort of sucks.

In my experience someone gets stoked on trying this out but once everything is all implemented it disappoints and the system or service it is apart of is a one off after future services use something more mature the next time around.

For scale I have used NSQ to handle millions of message a second and then for smaller scale AWS services like SQS can handle things much more reliably.

Re: An introduction to RabbitMQ

#219

Post by 411111111111111 »

tankerdude wrote: post #172
vasco wrote: post #144

Earlier quoted context omitted.

Celery can be backed by RabbitMQ, not sure if that's what you meant, but all of what you described can be abstracted away. I didn't have the same experiences with months taken to get up to speed. Moreover, at work RabbitMQ is probably our most stable underlying tool, perhaps toe to toe with Redis. And that's saying a lot, since I consider Redis to almost be a piece of art in how great of a tool it is. Back to RabbitM…

You would think, until you get to a split brain issue. The master and failover lose connectivity, and they each then think they're the master. There's ways to repair it (and it has happened to me one total time in 4 years), but it does happen. I personally try to make my message processing idempotent for the worker to help alleviate these situations.

haven't encountered it personally, so honest question here: how does a split brain situation become an issue in a message queue?

there are some possible situation from my naive viewpoint:

1. the 'active' queue keeps jumping between, consumers & producers keep reconnecting

=> everything is still consumed, but takes longer as producers write into alternating queues, which are consumed ... albeit slowly whenever the switch happens

2. they're database backed, so they'll try to write into the same table

=> usually software that does this (but cant handle several writers) also creates a `lock` which has to be manually reset before the failover can come up. if its reset, the other node would fail. only one is up, so no issue?

3. producers/consumers dont notice that the 'active' mq changed, and keep running on initial

=> issue manifests as soon as any system is restarted. but only slowly so you got time to handle it with minor service degradation

none of them really sound that bad to me -- but as i said before, i haven't encountered it before, so i might just overlooking something really obvious?

Re: An introduction to RabbitMQ

#220

Post by codeduck »

kabacha wrote: post #177

I'm really surprised to see so much positivity about rabbitmq here when it's probably the most sweared-at software in the space. Let me share my anecdote. In my last work place I got onboarded on rabbitmq and it was such a painful software to work with and almost impossible to set up locally that I silently sneaked in simple redis list as queue alternative for my dev environment. The whole rabbitmq and it's pika libr…

Amusingly, RabbitMQ is the one message broker that HASN'T given me grief in my career.
Post reply on HN