Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why do message queue-based architectures seem less popular now?

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 376 posts

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#111

Post by RangerScience »

kypro wrote: post #41
tuckerconnelly wrote: post #3

I can offer one data point. This is from purely startup-based experience (seed to Series A). A while ago I moved from microservices to monolith because they were too complicated and had a lot of duplicated code. Without microservices there's less need for a message queue. For async stuff, I used RabbitMQ for one project, but it just felt...old and over-architected? And a lot of the tooling around it (celery) just was…

In my experience monoliths don't reduce complexity, they just shift it. The main issue with monoliths is that they don't have clear and explicit separation of concern between domain concerns, therefore it's very easy for your monolith codebase to devolve into a mess of highly interconnected spaghetti code with time. This is especially true if you're building something large with a lot of developers who don't necessar…

"Cleverness, like complexity, is inevitable. The trick is making sure you're getting something worthwhile in exchange."

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#112

Post by k8sToGo »

steve1977 wrote: post #109
hn_throwaway_99 wrote: post #102

I like a lot of the answers, but something else I'd add: lots of "popular" architectures from the late 00s and early 2010s have fallen by the wayside because people realized "You're not Google. Your company will never be Google." That is, there was a big desire around that time period to "build it how the big successful companies built it." But since then, a lot of us have realized that complexity isn't necessary for…

> because people realized "You're not Google. Your company will never be Google." Is that also why almost no one is using microservices and Kubernetes?

Kubernetes brings more than just being Google. In a way it’s also an ecosystem.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#113

Post by mewpmewp2 »

KronisLV wrote: post #97
bigiain wrote: post #46

Earlier quoted context omitted.

Tracking, visitor analytics, and observability type things are all (in general) going out to 3rd party specialist services for those things, and getting dropped into a time series database (or, for us old school gray beards, a log file) and processed later. It's rare for the website devs to be doing anything more complex that adding in even more javascript to their pages for those, no need to message queues for that.…

> Order confirmation emails and sms messages are triggered by the order submission, and again usually sent off to a 3rd party bulk email or SMS service. Twilio or Campaign Monitor or Mailchimp will have queues and retry mechanisms, but again the website devs are just firing off an API call to some 3rd party that's dealing with that. In my case, I need to compile templates for the e-mails to be sent, which is somewhat…

What is taking so long for compiling?

And what language are you using? E.g. in NodeJS you can fire of requests without having ti have a new thread or having to wait for it.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#114

Post by onion2k »

bigiain wrote: post #18

My, perhaps overly cynical view, is that Message Queue architecture and blogging was all about "Resume Driven Development" - where almost everybody doing it was unlikely to ever need to scale past what a simple monolith could support running on a single laptop. All the same people who were building nightmare micro service disasters requiring tens of thousand of dollars a month of AWS services. These days all those pe…

Anything can be 'resume driven design'. If someone is rewarded with a raise, promotion, or even just praise sometimes, for applying a technology to a problem without being required to prove if the technology is appropriate they'll find a way to jam that tech into their domain regardless.

Sometimes that promotion is rewarded by going a different company while being able to say "yes, I used X in my previous role."

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#115

Post by steve1977 »

k8sToGo wrote: post #112
steve1977 wrote: post #109

Earlier quoted context omitted.

> because people realized "You're not Google. Your company will never be Google." Is that also why almost no one is using microservices and Kubernetes?

Kubernetes brings more than just being Google. In a way it’s also an ecosystem.

That‘s nice, but the question is if you (i.e. your company) needs this ecosystem.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#116

Post by akira2501 »

vishnugupta wrote: post #66

Speaking from my own experience message queues haven’t disappeared as much as have been abstracted away. For example enqueue to SQS + poll became invoke server less process. There is a message queue in there somewhere just that it’s not as exposed. Or take AWS SNS which IMO is one level of abstraction higher than SQS. It became so feature rich that it can practically replace SQS. What might have disappeared is those…

> There is a message queue in there somewhere just that it’s not as exposed.

It's still pretty exposed. You can set redelivery timeouts and connect a dead letter queue to lambda functions. Even just the lambda invoke API is obviously just a funny looking endpoint for adding messages to the queue.

> as much as have been abstracted away

In AWS in particular into EventBridge which further extends them with state machines. They've become the very mature corner stone of many technologies.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#117

Post by jiggawatts »

matrix87 wrote: post #88
kypro wrote: post #41

Earlier quoted context omitted.

In my experience monoliths don't reduce complexity, they just shift it. The main issue with monoliths is that they don't have clear and explicit separation of concern between domain concerns, therefore it's very easy for your monolith codebase to devolve into a mess of highly interconnected spaghetti code with time. This is especially true if you're building something large with a lot of developers who don't necessar…

Microservices also introduce the issue of maintaining schema compatibility for messages between services which usually leads to additional code in order to maintain backward compatibility From a technical POV they are good for horizontally scaling different workloads which have a different resource footprint From my experience, when a company decides to go the microservice route, it's more for the sake of solving an…

This presupposes that there is more than one line manager.

I see people trying to apply micro services architectures to a web app with a single developer.

As in literally taking a working monolith written by one person and having that one person split it up into tiny services.

It’s madness.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#118

Post by mewpmewp2 »

matrix87 wrote: post #88
kypro wrote: post #41

Earlier quoted context omitted.

In my experience monoliths don't reduce complexity, they just shift it. The main issue with monoliths is that they don't have clear and explicit separation of concern between domain concerns, therefore it's very easy for your monolith codebase to devolve into a mess of highly interconnected spaghetti code with time. This is especially true if you're building something large with a lot of developers who don't necessar…

Microservices also introduce the issue of maintaining schema compatibility for messages between services which usually leads to additional code in order to maintain backward compatibility From a technical POV they are good for horizontally scaling different workloads which have a different resource footprint From my experience, when a company decides to go the microservice route, it's more for the sake of solving an…

You could do that with code owners file in a monolith as well.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#119

Post by RangerScience »

primitivesuave wrote: post #61
kypro wrote: post #41

Earlier quoted context omitted.

In my experience monoliths don't reduce complexity, they just shift it. The main issue with monoliths is that they don't have clear and explicit separation of concern between domain concerns, therefore it's very easy for your monolith codebase to devolve into a mess of highly interconnected spaghetti code with time. This is especially true if you're building something large with a lot of developers who don't necessar…

Amazing this was downvoted. The comment starts with "in my experience" and is hardly a controversial perspective. I beg the HN community, stop disincentivizing people from respectively providing a converse opinion, lest this become yet another echo chamber of groupthink.

It relates that there was experience, but not what that experience was - We can read and understand that they're reporting their own experience, but that's about it.

One could say "In my experience, the earth is flat", but there's not much a conversation to be had there.

One could say, "In my experience, the earth is flat - I got in my car, went for a drive in one direction, and eventually hit a cliff at the ocean instead of finding myself back where I started". Now there's something to talk about.

(To be clear: this is the internet, and a limited communication medium: I'd assume OP could relate details about their experience, and it's totally reasonable that instead of taking the time to do that, they went outside and touched grass)

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#120

Post by jsumrall »

I’m concerned that a lot of commenters don’t appreciate the difference between a queue and a log. Kafka is not a queue.

I think like most have said is that it’s just not a popular topic anymore to blog about but it’s still used. OTOH logs like Kafka have become more ubiquitous. Even new and exciting systems like Apache Pulsar (a log system that can emulate a queue) have implemented the Kafka API.

Post reply on HN