Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An introduction to RabbitMQ

erlang-solutions.com

Post reply on HN
Print view View original

161–170 of 263 posts

Re: An introduction to RabbitMQ

#161

Post by akoncius »

ketralnis wrote: post #87
dkersten wrote: post #39

Earlier quoted context omitted.

How is it for production deployment? I was considering it for something recently, but got overwhelmed by the documentation on setting up a fault-tolerant production deployment, so have been avoiding it. Was this an overreaction? What is your experience with that? Also, do you happen to know how well it works in a fault-tolerant way for communicating between services that are in different data centers? My main use-cas…

We use it for more or less everything at reddit. Almost every user action corresponds to a rabbit queue

sounds cool! how big queues are on your setup? how big mq instances (servers) are? do you use HA, replications/failovers?

Re: An introduction to RabbitMQ

#162

Post by robbiep »

mythrwy wrote: post #153
vasco wrote: post #144

Earlier quoted context omitted.

Celery can be backed by RabbitMQ, not sure if that's what you meant, but all of what you described can be abstracted away. I didn't have the same experiences with months taken to get up to speed. Moreover, at work RabbitMQ is probably our most stable underlying tool, perhaps toe to toe with Redis. And that's saying a lot, since I consider Redis to almost be a piece of art in how great of a tool it is. Back to RabbitM…

I have simple single node deployment and I was floored how easy it was to set up with Celery. Really surprised. I was kicking myself for not using it sooner. Granted I don't know all the intricacies of RabbitMQ and this was just one step beyond os.popen, but it was painless, like half an hour painless to set up and it has worked really well. *edit: reading some of the other posts now I'm waiting for the other shoe to…

I also got my first queue set up and running within a reasonable period of time with celery. I have no idea of the internals of RabbitMQ and took longer with celery really (back on python 2.7) but that system has been in prod for 6 years now without really needing any maintenance

Re: An introduction to RabbitMQ

#163

Post by neeleshs »

zonk_ wrote: post #50
dkersten wrote: post #39

Earlier quoted context omitted.

How is it for production deployment? I was considering it for something recently, but got overwhelmed by the documentation on setting up a fault-tolerant production deployment, so have been avoiding it. Was this an overreaction? What is your experience with that? Also, do you happen to know how well it works in a fault-tolerant way for communicating between services that are in different data centers? My main use-cas…

We use it in a fairly big scale for our slack bot system. It was just set up once, as akyu said, and since then it just works. Whenever we had troubles, it was always anything other than RabbitMQ. I've also looked into other solutions (ActiveMQ, Google PubSub, ...) and RabbitMQ is by far the most straight-forward and quick to set up. There are some edge cases that it doesn't cover as well, for example automatic retri…

We use Google Pub-Sub and got the whole thing up and running very quickly with Spring integration. Message durability, automatic consumer load balancing, automatic retries, some easy broadcast patterns - all out of the box and literally a click of a button on the infra side.

Has worked out quite well so far.

Re: An introduction to RabbitMQ

#164

Post by jv22222 »

webscalist wrote: post #117

RabbitMQ has huge learning curve if you're trying to build a worker queue. First, you'll learn about ack/noack and get the worker ack on success. Then, you'll learn about dead letter queue ... etc for delayed retries. Now, you'll have a topic exchange and a bit hairy routing in place using wildcards. And you mistakenly set dead letter routing key so that expired messages end up in multiple queues (retry queues and ac…

The biggest annoyance I found with RabbitMQ was that it could take up to 10-15 mins to restart if it had a lot of jobs.

This was back in 2015 - might be better now.

Re: An introduction to RabbitMQ

#165

Post by crad »

LeonM wrote: post #138
crad wrote: post #38

Using the opportunity to pimp my book, RabbitMQ in Depth: https://www.manning.com/books/rabbitmq-in-depth :)

So weird seeing you post that, as I literally have this book on my desk right now. Thanks Gavin, I learned a lot from reading it!

That's awesome! I'm glad it was useful!

Re: An introduction to RabbitMQ

#166

Post by lukebakken »

jsmith45 wrote: post #131
lukebakken wrote: post #76

Earlier quoted context omitted.

> Especially with the .NET implementation we had quite a lot of trouble. Its not following current .NET patterns and has strange quirks. It would be great to get specific, actionable feedback with your experience, either via a message to the rabbitmq-users mailing list or via a GitHub. The .NET client is an old library but considerable effort into improvement went into version 6.0. The plan for 7.0 is to address old…

The biggest issues are the public API surface. If the library were being designed from scratch today, pretty much every method on the model would be Async. After all, if it leads to any network I/O of any kind, that can block. Working with the current public API, Trying to implement a publish wrapper that never blocks, and returns a task that either completes when the publisher confirm is received, or faults after so…

Thanks for taking the time to respond. I created this issue so that this feedback is not lost - https://github.com/rabbitmq/rabbitmq-dotnet-client/issues/84...

If you have code you can share that you used to address shortcomings in the client, we could get ideas from it for the next major release. Cheers!

Re: An introduction to RabbitMQ

#167

Post by keithnz »

webscalist wrote: post #117

RabbitMQ has huge learning curve if you're trying to build a worker queue. First, you'll learn about ack/noack and get the worker ack on success. Then, you'll learn about dead letter queue ... etc for delayed retries. Now, you'll have a topic exchange and a bit hairy routing in place using wildcards. And you mistakenly set dead letter routing key so that expired messages end up in multiple queues (retry queues and ac…

weird, I haven't done much digging in to the details of RabbitMQ, but I integrated it in a matter of hours, and have it deployed in production systems (for quite sometime now) and it works really solidly. I haven't tried to get too clever though.

Re: An introduction to RabbitMQ

#168

Post by rlander »

webscalist wrote: post #117

RabbitMQ has huge learning curve if you're trying to build a worker queue. First, you'll learn about ack/noack and get the worker ack on success. Then, you'll learn about dead letter queue ... etc for delayed retries. Now, you'll have a topic exchange and a bit hairy routing in place using wildcards. And you mistakenly set dead letter routing key so that expired messages end up in multiple queues (retry queues and ac…

I have my own share of objections, mainly concerning the over-engineered nature of RabbitMQ, but most of the “huge learning curve” items that you’ve described can be learned in an afternoon by a motivated software engineer. Besides, she will have to learn those concepts anyway because they apply to most brokers.

Re: An introduction to RabbitMQ

#169

Post by EdwardDiego »

halfmatthalfcat wrote: post #13

I've been trying to rationalize using either RabbitMQ or Kafka for something I'm building. High messages per second but with more complex routing topologies. Rabbit seems to be the right path but I'm worried about scaling out as many sources seem to point as Kafka being more scalable (at least horizontally). I've been looking into Rabbit's Federation but it's still not clear if that will solve the problem down the ro…

Look into Pulsar, it can function as a message queue or pub/sub like Kafka.

By default it only retains non-acked messages, multiple subscription modes, can use non-persistent messaging, dead letter queue, scheduled delivery, can use Pulsar Functions to implement custom routing etc.

Scales like Kafka (probably better) and has cluster replication built in.

Re: An introduction to RabbitMQ

#170

Post by anon176 »

keithnz wrote: post #167
webscalist wrote: post #117

RabbitMQ has huge learning curve if you're trying to build a worker queue. First, you'll learn about ack/noack and get the worker ack on success. Then, you'll learn about dead letter queue ... etc for delayed retries. Now, you'll have a topic exchange and a bit hairy routing in place using wildcards. And you mistakenly set dead letter routing key so that expired messages end up in multiple queues (retry queues and ac…

weird, I haven't done much digging in to the details of RabbitMQ, but I integrated it in a matter of hours, and have it deployed in production systems (for quite sometime now) and it works really solidly. I haven't tried to get too clever though.

You must of followed a good guide on getting it setup. Took me 2 days to get it solid. Then we decided to just use redis.
Post reply on HN