Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why do message queue-based architectures seem less popular now?

news.ycombinator.com

Post reply on HN
Print view View original

321–330 of 376 posts

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#321

Post by coffeecloud »

EVa5I7bHFq9mnYK wrote: post #198
busterarm wrote: post #162

Going to give the unpopular answer. Queues, Streams and Pub/Sub are poorly understood concepts by most engineers. They don't know when they need them, don't know how to use them properly and choose to use them for the wrong things. I still work with all of the above (SQS/SNS/RabbitMQ/Kafka/Google Pub/Sub). I work at a company that only hires the best and brightest engineers from the top 3-4 schools in North America a…

Because JS is single threaded, and an average programmer today is too dumb to learn any language beyond JS, you must build everything into "microservices".

You sound like a joy to work with

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#324

Post by jiggawatts »

baq wrote: post #159
jiggawatts wrote: post #117

Earlier quoted context omitted.

This presupposes that there is more than one line manager. I see people trying to apply micro services architectures to a web app with a single developer. As in literally taking a working monolith written by one person and having that one person split it up into tiny services. It’s madness.

If your goal is to learn kubernetes instead of developing a product, then go for it IMHO, no better way. Just make sure everyone is on board with the idea.

I call this customer-funded self-training.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#325

Post by therealdrag0 »

robertclaus wrote: post #278

I find it super interesting that the comments calling out "obviously we all still use message queues and workers, we just don't write about them" are buried half way down the comments section by arguments about Microservices and practical scalability. A junior engineer reading the responses could definitely get the false impression that they shouldn't offload heavy computation from their web servers to workers at all…

It’s blowing my mind. Queues are the most bog standards architectural component.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#326

Post by avikalp »

Nice observation. I still use Google Pub/Sub in my application - I recently also gave a talk on how we use Pub/Sub for our use case in a GCDG event (GCDG stands for Google Cloud Developer Group).

But now that I think about it, we don't use it in the traditional sense. Most of our regular operations work well enough by just using the "async" pattern in our programming (in JS and Rust).

The only place we use Pub/Sub is for communication between our NodeJS backend server and the Rust servers that we deploy on our client's VMs. We didn't want to expose a public endpoint on the Rust server (for security). And there was no need for a response from the Rust servers when the NodeJS server told it to do anything.

We don't fully utilize the features of a messaging queue (like GCP's Pub/Sub), but there just wasn't a better way for our specific kind of communication.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#327

Post by XajniN »

RangerScience wrote: post #119
primitivesuave wrote: post #61

Earlier quoted context omitted.

Amazing this was downvoted. The comment starts with "in my experience" and is hardly a controversial perspective. I beg the HN community, stop disincentivizing people from respectively providing a converse opinion, lest this become yet another echo chamber of groupthink.

It relates that there was experience, but not what that experience was - We can read and understand that they're reporting their own experience, but that's about it. One could say "In my experience, the earth is flat", but there's not much a conversation to be had there. One could say, "In my experience, the earth is flat - I got in my car, went for a drive in one direction, and eventually hit a cliff at the ocean in…

That’s not a reason to downvote. Downvoting is a censorship tool.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#328

Post by namaria »

bjourne wrote: post #314
alfiedotwtf wrote: post #180

Earlier quoted context omitted.

This! You NEEDED to scale horizontally because machines were just doing too much. I remember when our Apache boxes couldn’t even cope doing SSL so we had a hardware box doing it on ingress!

15 years ago I ran a website (Django+Postgres+memcached) serving 50k unique daily visitors on a dirt cheap vps. Even back then the scalability issues were overstated.

As the stock market prices now expected future growth, architecture had to justify rising stock prices by promising future scalability.

It was never about the actual workloads, much more about growth projections. And a whole lot of cargo cult behavior.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#329

Post by koliber »

Message queues are often chosen as a communication protocol in microservice-based architectures. Microservices were a fad and people have sobered up. People have learned when microservices deliver a benefit, and when they are unnecessary. In many cases, they are unnecessary.

Queues are still very useful for queueing up asynchronous work. Most SaaS apps I've worked with use one. However, there is a difference to what kind of queue you need to queue a few thousand tasks per day, vs using the queue as the backbone of all of your inter-service communications. For the first use case, using a DB table or Redis as a queue backend is often enough.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#330

Post by koliber »

ithkuil wrote: post #156
hn_throwaway_99 wrote: post #102

I like a lot of the answers, but something else I'd add: lots of "popular" architectures from the late 00s and early 2010s have fallen by the wayside because people realized "You're not Google. Your company will never be Google." That is, there was a big desire around that time period to "build it how the big successful companies built it." But since then, a lot of us have realized that complexity isn't necessary for…

We also have much much bigger single machines available for reasonable money. So a lot of reasonable workloads can fit in one machine now that used to require a small cluster

This was true in the 2000's and 2010's as well. A lot of the work could be handled by a single monolithic app running on one or a small handful of servers. However, because of the microservices fad, people often created complicated microservices distributed across auto-scaling kubernetes clusters, just for the art of it. It was unneeded complexity then, as it is now, in the majority of cases.
Post reply on HN