We've used RabbitMQ since 2010 in KAZOO. I would argue, save one or two instances in the intervening 10 years, that RabbitMQ is the most stable piece of the infrastructure. I think it might be the only open-source project we build on that we haven't committed upstream to because we haven't encountered any issues in our usage.
An introduction to RabbitMQ
31–40 of 263 posts
Re: An introduction to RabbitMQ
#32Using it in multiple projects: The software itself is great and provides great value. Only pitfall are the available libs. Especially with the .NET implementation we had quite a lot of trouble. Its not following current .NET patterns and has strange quirks. Does anyone know a good alternative to the "official" one?
MassTransit is great, the maintainer is very active on Discord, and since quarantine has been streaming every Thursday night (for my UTC-5 anyway). Documentation quality has increased greatly the last few months as well.
Re: An introduction to RabbitMQ
#33Earlier quoted context omitted.
I do and it's great. It doesn't have some MQTT features sush as persisted messages or QOS 2 but if you don't need that, it's a fine MQTT broker.
Wait, what are you talking about? RabbitMQ does have persistent messages, you just need to set the queue as "durable", and the messages persist even during failures.
Indeed I'm a bit confused. I remembered about having to find a workaround because I couldn't use retained messages. It's actually not working only for subscribers with wildcards : https://github.com/rabbitmq/rabbitmq-mqtt/issues/154
Re: An introduction to RabbitMQ
#34RabbitMQ is great. One of the few pieces of software I've used that "just works". The only downside is once you get message-queue-pilled, you start seeing opportunities to refactor/redesign with message queues everywhere and it can be hard to resist the urge. It really is remarkable how, when used appropriately, message queues can dramatically simplify a system.
+1. Discovered RabbitMQ/AMQP around 2010, since then tech went through a 2015-era wave of HTTP microservices that has come, and, largely gone or moved to MQ.
When you say "gone or moved to MQ" - if not moved to messaging services like RabbitMQ/NATS/etc, where else could things have gone? At least from my experience, HTTP microservices are still very common, especially when using things like AWS Lambdas.
I feel like most continually-running backends will make use of RabbitMQ/NATS/ZeroMQ/etc, or more and more I see lightweight systems going completely serverless and just using lambdas - which are HTTP microservices.
Re: An introduction to RabbitMQ
#35I've been trying to rationalize using either RabbitMQ or Kafka for something I'm building. High messages per second but with more complex routing topologies. Rabbit seems to be the right path but I'm worried about scaling out as many sources seem to point as Kafka being more scalable (at least horizontally). I've been looking into Rabbit's Federation but it's still not clear if that will solve the problem down the ro…
Rabbit's federation is a good way to bridge point-to-point connections between geographically distributed systems. I'm not sure that's a great scaling pattern for throughput though.
The clustering might look tempting but it hasn't been resilient for me in the face of janky networks. Split brains and data loss can result.
In the past I've scaled my rabbits for throughput by implementing my own routing/sharding layer.
If you're tempted to use the message persistence and you care about retaining messages, kafka is a bigger but much more capable hammer.
Re: An introduction to RabbitMQ
#36I've been trying to rationalize using either RabbitMQ or Kafka for something I'm building. High messages per second but with more complex routing topologies. Rabbit seems to be the right path but I'm worried about scaling out as many sources seem to point as Kafka being more scalable (at least horizontally). I've been looking into Rabbit's Federation but it's still not clear if that will solve the problem down the ro…
Author here. I've seen quite a lot messages going through RabbitMQ. I wouldn't worry too much about scaling, because the possibilities depend very much on the architecture. With some tuning RabbitMQ can take you a long way. I would give clustering a go and see where the limits are before exploring more complicated architectures like federation.
Could you explain how RabbitMQ clustering is going to improve performance? For how it works I would expect it to lower performance.
Re: An introduction to RabbitMQ
#37I used RabbitMQ together with python and celery quite extensively and it scales really well. One thing we had trouble with though was to find a nice mechanism to scheduled tasks. Eg. “Run this task 12 hours before departure”. Maybe AMQP is the wrong place to solve that problem.
Re: An introduction to RabbitMQ
#38Using the opportunity to pimp my book, RabbitMQ in Depth: https://www.manning.com/books/rabbitmq-in-depth
:)
Re: An introduction to RabbitMQ
#39RabbitMQ is great. One of the few pieces of software I've used that "just works". The only downside is once you get message-queue-pilled, you start seeing opportunities to refactor/redesign with message queues everywhere and it can be hard to resist the urge. It really is remarkable how, when used appropriately, message queues can dramatically simplify a system.
How is it for production deployment? I was considering it for something recently, but got overwhelmed by the documentation on setting up a fault-tolerant production deployment, so have been avoiding it. Was this an overreaction? What is your experience with that?
Also, do you happen to know how well it works in a fault-tolerant way for communicating between services that are in different data centers?
My main use-case is to receive status/change notifications from a service running elsewhere from the API server servicing the UI, in order to avoid polling for new data.
Re: An introduction to RabbitMQ
#40Would anyone be able to explain the benefits of RabbitMQ over NATS? As far as I've seen, it's really just that RabbitMQ is more feature-rich, which I personally feel like isn't that crucial, as frankly many systems are not going to take advantage of those more complex functionalities anyway.