Ask HN: Why do message queue-based architectures seem less popular now?

In the late 2000s and early 2010s, I remember seeing lots of hype around building distributed systems using message queues (e.g. Amazon SQS, RabbitMQ, ZeroMQ, etc.) A lot of companies had blog posts highlighting their use of message queues for asynchronous communication between nodes, and IIRC the official AWS design recommendations at the time pushed SQS pretty heavily.

Now, I almost never see engineering blog posts or HN posts highlighting use of message queues. I see occasional content related to Kafka, but nothing like the hype that message queues used to have.

What changed? Possible theories I'm aware of:

* Redis tackled most of the use-case, plus caching, so it no longer made sense to pay the operational cost of running a separate message broker. Kafka picked up the really high-scale applications.

* Databases (broadly defined) got a lot better at handling high scale, so system designers moved more of the "transient" application state into the main data stores.

* We collectively realize that message queues-based architectures don't work as well as we hoped, so we build most things in other ways now.

* The technology just got mature enough that it's not exciting to write about, but it's still really widely used.

If people have experience designing or implementing greenfield systems based on message queues, I'd be curious to hear about it. I'd also be interested in understanding any war stories or pain points people have had from using message queues in production systems.

I can offer one data point. This is from purely startup-based experience (seed to Series A).

A while ago I moved from microservices to monolith because they were too complicated and had a lot of duplicated code. Without microservices there's less need for a message queue.

For async stuff, I used RabbitMQ for one project, but it just felt...old and over-architected? And a lot of the tooling around it (celery) just wasn't as good as the modern stuff built around redis (bullmq).

For multi-step, DAG-style processes, I prefer to KISS and just do that all in a single, large job if I can, or break it into a small number of jobs.

If I REALLY needed a DAG thing, there are tools out there that are specifically built for that (Airflow). But I hear they're difficult to debug issues in, so would avoid at most costs.

I have run into scaling issues with redis, because their multi-node architectures are just ridiculously over-complicated, and so I stick with single-node. But sharding by hand is fine for me, and works well.

Like micro-services, it's not a bad idea, but it's not the hammer for every nail as people who write books and blog posts seem to push for (they've now moved on past blockchain to AI).

If you have a problem that logically has different async services, sure, use Redis or something. Databases also were able to handle this problem, but weren't as sexy, and they explicitly handle the problem better now. Just another tool in the toolbelt.

NoSQL was another solution in search of problems, but databases can handle this use-case better now too.

Hype died down along with "microservices" where queues made more sense.

We all learned that a distributed monolith is worse than just having a monolith. Truly independent event-based systems still are very useful, but not when they have to communicate stuff back and forth to achieve a single problem.