201–210 of 263 posts
It's working well for us but we occasionally get blips where for very short periods of time messages get "stuck" in between application code events on different servers and we cannot figure out why. It's very rare. Maybe a burst of 5 messages every 10 million messages. Any ideas on how to even debug this type of thing? Help! We think it might be a tcp connection failure but we have no idea.
I've had truly terrible experiences with RabbitMQ. I believe that it should not be used in any application where message loss is not acceptable. Its two big problems are that it cannot tolerate network partitions (reason enough to never use it in production systems, see https://twitter.com/antifuchs/status/735628465924243456 ), and it provides no backpressure to producers when it starts running out of memory. In my l…
This worked fine until things got behind and then we couldn't keep up. We were able to work around that by using a hashed exchange that spread messages across 4 queues. It hashed based on timestamp inserted by a timestamp plugin. Since all operations for a queue happen in the same event loop, any sort of backup led to pub and sub operations fighting for CPU time. By spreading this across 4 queues we wound up with 4x the CPU capacity for this particular exchange. With 2000 queues you probably didn't run into that issue very often.
My general problem is that it's really hard to figure out which architecture is right for which system. There's a different architecture for: * one queue with billions of messages * a millions of queues with small numbers of messages per queue * many queues with many messages per queue There are also different topologies: * Anyone can send a message to anyone (O(n^2) queues) * One publisher with millions of subscribe…
Likewise this configurability makes case specific benchmarks very awkward.
Use RabbitMQ for a call center handling thousands of calls per second. It worked fine integrated with Flower, Celery and Python...but once we went production, became a black box which every setting was hard to find documentation or support, we ended up having to build huge Machines with tons of memory and CPU and still saw messages lost no explanation. Ended up moving to PubSub and rebuild the whole app
I'm very well versed on RabbitMQ. We use it internally in a .NET codebase. Anyone considering RabbitMQ needs to read up on "network partitions", how to build your cluster to avoid them (odd number of nodes and pause_minority), your recovery strategy for when a network partition occurs (it will occur), your personal/organizational tolerance for message loss and a plan for how you will upgrade your cluster at some late…
If you're using RabbitMQ on .Net, I highly recommend using NserviceBus. It's made working with queues so easy . It handles maintaining a connection and retrying/acknowledging messages for you.
Earlier quoted context omitted.
As someone who has ran a number of messaging systems in production, this is what my current take is in general: If you are moving to a more "event-sourced" architecture, usually two main concerns (beyond basic operational stuff of uptime, scale, etc) are routing and long-term retention. RabbitMQ has the routing but not the retention. Kafka can have the retention and the routing, but it can be complex/expensive. Apach…
Would you mind expanding on some of the operational complexity you ran into with pulsar? I think pulsar is wonderful, but I haven't had the chance to use it for anything serious / in production yet, so I'm curious what pain points you had.
Anyone know of any Pulsar hosting providers?
Using this opportunity to shout out to Rascal ( https://github.com/guidesmiths/rascal ) which makes using RabbitMQ on Node an absolute joy.
Same with MassTransit[0] and .NET. We have several distributed .NET Core services running in our data center, services running on employee PCs, etc all communicating via RMQ with MassTransit and it's great. The primary maintainer is very active (streams every Thursday evening) and the documentation has gone from "pretty bad" to really good in the last few months. [0] https://masstransit-project.com/
Would anyone be able to explain the benefits of RabbitMQ over NATS? As far as I've seen, it's really just that RabbitMQ is more feature-rich, which I personally feel like isn't that crucial, as frankly many systems are not going to take advantage of those more complex functionalities anyway.
I've got a connection/channel question for those who have built solutions with rabbitmq-- how did you decide as to how many connections and channels-per-connection to use? Does connection pooling even make sense for RabbitMQ? My impression is that channel pooling may make more sense. Thoughts?
An application usually has one connection, and many channels. Our pattern is to dedicate one channel for all publishing and then N channels mapped to consumer threads. You don't have to pool connections as channels are multiplexed by them. Things to watch out for: - opening too many channels - these map to Erlang processes and can overwhelm your server if you go over ulimits - sharing consumer channels between thread…
The general takeaway from this should be: if you've got a particular stream of messages (either a producer or a consumer) that pushes many thousands or even tens of thousands of messages per second, use a separate TCP connection. For anything else that is slower (dozens of messages per second), multiple channels on the same connection work great.
One last consideration is that when a given channel misbehaves or you perform an operation that the broker doesn't like, the only recovery that I've seen is to shut down the entire connection which can affect others channels on the same connection.