Live data from Hacker News

RabbitMQ 4.0

github.com

31–40 of 120 posts

Re: RabbitMQ 4.0

#31
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

How do you manage schema for exchanges, queues, and routing? The pattern seems to be for each client to set up the schema for the parts they touch before hand, but that doesn't scale well. The schema ends up siloed in each project and nobody knows what the current state should be.

Clients creating server topologies does scale very well with the new Khepri metadata store in RabbitMQ 4.0. Applications having an intimate relationship with their middleware (RabbitMQ) and creating flexible routing topologies is one of main strengths of RabbitMQ! RabbitMQ also allows operators to import (exchange, queue, and binding) definitions on boot. Server topologies can nowadays even be declared via YAML files in Kubernetes (https://github.com/rabbitmq/messaging-topology-operator). This way all the desired state is in one single place and the Kubernetes operator reconciles such that this declaratively declared desired schema is created within RabbitMQ.

Re: RabbitMQ 4.0

#33
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

The real reason people don't use it is because they don't know about it or understand it. Then they apply the "it doesn't scale" retroactively. You have to read a lot of docs or you WILL hold RabbitMQ wrong.

I agree, but there are a lot of footguns with RMQ. A great example of one is that you'll slow down your cluster by adding more RMQ servers (something that's bit us in the past). Which is a forgivable mistake as most people would expect that more cores == faster RMQ. (For RMQ, that doesn't work because Durable messages need to be replicated to the other nodes in the cluster. More nodes == more replication)

The ideal RMQ cluster has 3 servers and is dedicated to just a few apps.

Re: RabbitMQ 4.0

#34
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

Is there a big advantage over Redis? Just coming from the Ruby world I know Sidekiq is extremely popular as a Redis backed queue. I know there's a RabbitMQ backed queue called Sneakers that's gained a lot of popularity as well though.

Just wondering what the biggest selling points are for somebody making a decision?

Re: RabbitMQ 4.0

#35
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

Modern RabbitMQ scales well. RabbitMQ Streams provide a throughput of millions of messages per second. With native support for MQTT, more than a million users can connect to RabbitMQ: https://www.rabbitmq.com/blog/2023/03/21/native-mqtt

The new Khepri metadata store in RabbitMQ 4.0 allows for even better scalability.

Re: RabbitMQ 4.0

#36
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

Do you know of any book or video to get started with Rabbit?

Earlier this year I tried setting it up as a websockets message broker in a Spring Boot app but failed miserably. I ended up using Spring’s simple broker.

Re: RabbitMQ 4.0

#37
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

Do you have any recommended resources to learn how to apply these tools (RabbitMQ, Nats, etc) to typical enterprise services? Common patterns/usecases/best practices and things?

Re: RabbitMQ 4.0

#39
post #3

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great. Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

whut.. transferred TB of data per hour with Rabbit.. does it not scale anymore?

Re: RabbitMQ 4.0

#40
post #12

Earlier quoted context omitted.

I've never used RabbitMQ, but NATS supports clustering, super clustering, and leaf node connections. I'm guessing the latter is the closest to what would be considered "federation" in this context. Edit: spelling

None of them are close. RabbitMQ has traffic shaping, MQ->MQ routing (federation), and all sorts of stuff that is super important if you use it as a generalized routing system (vs plain IP). NATS doesn't have that and it's almost certainly firmly out of scope.

When you say "MQ->MQ routing (federation)" I have a hard time understanding how that is not close to leaf nodes. Leaf nodes allow independently operated / managed NATS servers to share data back and forth, which to me is what federation is about. Do you have any resources you recommend to help me grasp why this is different?
Post reply on HN