Message Queues: A Simple Guide with Analogies (2024)
cloudamqp.com
Message Queues: A Simple Guide with Analogies (2024)
1–10 of 29 posts
Re: Message Queues: A Simple Guide with Analogies (2024)
#2While queues definitely play an important role in microservices architecture, I think it’s worth clarifying that they’re not unique to it. A queue can fit perfectly in a monolith depending on the use case. I regularly use queues for handling critical operations that might require retrying, for having better visibility into failed jobs, ensuring FIFO guarantees, and more. Queues are such a useful tool for building any resilient architecture that framing them as primarily a microservices concern might cause unnecessary confusion.
Re: Message Queues: A Simple Guide with Analogies (2024)
#3While queues definitely play an important role in microservices architecture, I think it’s worth clarifying that they’re not unique to it. A queue can fit perfectly in a monolith depending on the use case. I regularly use queues for handling critical operations that might require retrying, for having better visibility into failed jobs, ensuring FIFO guarantees, and more. Queues are such a useful tool for building any…
Totally agree. Banks use durable queues a lot to make sure things get processed. Or at least they used to.
Re: Message Queues: A Simple Guide with Analogies (2024)
#4While queues definitely play an important role in microservices architecture, I think it’s worth clarifying that they’re not unique to it. A queue can fit perfectly in a monolith depending on the use case. I regularly use queues for handling critical operations that might require retrying, for having better visibility into failed jobs, ensuring FIFO guarantees, and more. Queues are such a useful tool for building any…
Totally agree. Banks use durable queues a lot to make sure things get processed. Or at least they used to.
The analogy could be: “Queues are the like the todos list of your team. The todo item (message) stays there until it is successfully completed. It can be handled by the producer (monolith) or it can be handled by someone else (microservices).”
Re: Message Queues: A Simple Guide with Analogies (2024)
#5This is surprisingly basic knowledge for ending up on the front page.
It’s a good intro, but I’d love to read more about when to know it’s time to replace my synchronous inter service http requests with a queue. What metrics should I consider and what are the trade offs. I’ve learned some answers to this question over time, but these guys are theoretically message queue experts. I’d love to learn about more things to look out for.
There are also different types of queues/exchanges and this is critical depending on the types of consumer or consumers you have. Should I use direct, fan out, etc?
The next interesting question is when should I use a stream instead of a queue, which RabbitMQ also supports.
My advice, having just migrated a set of message queues and streams from AWS(AvtiveMQ) to RabbitMQ is think long and hard before you add one. They become a black box of sorts and are way harder to debug than simple HTTP requests.
Also, as others have pointed out, there are other important use cases for queues which come way before microservice comms. Async processing to free up servers is one. I’m surprised none of these were mentioned.
Re: Message Queues: A Simple Guide with Analogies (2024)
#6While queues definitely play an important role in microservices architecture, I think it’s worth clarifying that they’re not unique to it. A queue can fit perfectly in a monolith depending on the use case. I regularly use queues for handling critical operations that might require retrying, for having better visibility into failed jobs, ensuring FIFO guarantees, and more. Queues are such a useful tool for building any…
Monoliths also have to scale to multiple servers eventually, so message queues are an important architectural component to understand regardless of the organization of your services.
Re: Message Queues: A Simple Guide with Analogies (2024)
#7This is surprisingly basic knowledge for ending up on the front page. It’s a good intro, but I’d love to read more about when to know it’s time to replace my synchronous inter service http requests with a queue. What metrics should I consider and what are the trade offs. I’ve learned some answers to this question over time, but these guys are theoretically message queue experts. I’d love to learn about more things to…
> This is surprisingly basic knowledge for ending up on the front page.
Nothing wrong with that! Hacker News has a large audience of all skill levels. Well written explainers are always good to share, even for basic concepts.
Re: Message Queues: A Simple Guide with Analogies (2024)
#8This is surprisingly basic knowledge for ending up on the front page. It’s a good intro, but I’d love to read more about when to know it’s time to replace my synchronous inter service http requests with a queue. What metrics should I consider and what are the trade offs. I’ve learned some answers to this question over time, but these guys are theoretically message queue experts. I’d love to learn about more things to…
> This is surprisingly basic knowledge for ending up on the front page. Nothing wrong with that! Hacker News has a large audience of all skill levels. Well written explainers are always good to share, even for basic concepts.
Agree! In fact, I would appreciate more well written articles explaining basic concepts on the front page of Hacker News. It is always good to revisit some basic concepts, but it is even better to relearn them. I am surprised by how often I realize that my definition of a concept is wrong or just superficial.
Re: Message Queues: A Simple Guide with Analogies (2024)
#9While queues definitely play an important role in microservices architecture, I think it’s worth clarifying that they’re not unique to it. A queue can fit perfectly in a monolith depending on the use case. I regularly use queues for handling critical operations that might require retrying, for having better visibility into failed jobs, ensuring FIFO guarantees, and more. Queues are such a useful tool for building any…
Absolutely, 100%.
I work on PeopleSoft Enterprise Resource Planning applications - the "boring" back-office HR, Pay, Financials, Planning etc stuff.
The core architecture is late 80s - mid 90s. Couple of big architectural changes when internet/browsers and then mobile really hit. But fundamentally it's a very legacy / old school application. Lots of COBOL, if that helps calibrate :->
We use queues pervasively. It's PeopleSoft's preferred integration method for other external applications, but over the years a large number of internal plumbing is now via queues as well. PeopleSoft Integration Broker is kind of like an internal proprietary ESB. So understanding queues and messaging is key to my PeopleSoft Administrator teams wherever I go (basically sysadmins in service of PeopleSoft application:).
Re: Message Queues: A Simple Guide with Analogies (2024)
#10This is surprisingly basic knowledge for ending up on the front page. It’s a good intro, but I’d love to read more about when to know it’s time to replace my synchronous inter service http requests with a queue. What metrics should I consider and what are the trade offs. I’ve learned some answers to this question over time, but these guys are theoretically message queue experts. I’d love to learn about more things to…
> This is surprisingly basic knowledge for ending up on the front page. Nothing wrong with that! Hacker News has a large audience of all skill levels. Well written explainers are always good to share, even for basic concepts.
In principle, I agree, but “a message queue is… a medium through which data flows from a source system to a destination system” feels like a truism.