Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why do message queue-based architectures seem less popular now?

news.ycombinator.com

Post reply on HN
Print view View original

171–180 of 376 posts

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#171

Post by salomonk_mur »

flir wrote: post #150
appplication wrote: post #93

Earlier quoted context omitted.

To your comment on Airflow, I’ve been around that block a few times. I’ve found Airflow (and really any orchestration) be the most manageable when it’s nearly devoid of all logic to the point of DAGs being little more than a series of function or API calls, with each of those responsible for managing state transfer to the next call (as opposed to relying on orchestration to do so). For example, you need some ETL to h…

Ok, so question (because I really like the DAG approach in principle but don't have enough experience to have had my fingers burned yet): The way you use Airflow, what advantage does it have over crontab? Or to put it another way, once you remove the pipeline logic, what's left?

Observability is a huge upside.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#172

Post by okokwhatever »

pm90 wrote: post #28

They have become boring so there are less blogs about them. Thats good. The documentation for eg RabbitMQ is much better and very helpful. People use it as a workhorse just like they use Postgres/MySQL. There’s not much surprising behavior needed to architect around etc. I love boring software.

Agree

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#173

Post by benterix »

steve1977 wrote: post #109
hn_throwaway_99 wrote: post #102

I like a lot of the answers, but something else I'd add: lots of "popular" architectures from the late 00s and early 2010s have fallen by the wayside because people realized "You're not Google. Your company will never be Google." That is, there was a big desire around that time period to "build it how the big successful companies built it." But since then, a lot of us have realized that complexity isn't necessary for…

> because people realized "You're not Google. Your company will never be Google." Is that also why almost no one is using microservices and Kubernetes?

Define "no one". If you mean small shops, maybe. If you mean large organizations, I haven't seen even one in the last 5 years that wouldn't use them in one way or another.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#175

Post by benterix »

Kirth wrote: post #166
steve1977 wrote: post #115

Earlier quoted context omitted.

That‘s nice, but the question is if you (i.e. your company) needs this ecosystem.

Genuine question: say you have 3-4 services and a bunch of databases that make up your product, what's the alternative to plemping them all into K8s according to you?

If you use AWS, it's probably easier to use ECS that takes away some of the complexity from you.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#176

Post by busterarm »

otabdeveloper4 wrote: post #157
slau wrote: post #143

Earlier quoted context omitted.

Well, the one thing they have in common is that both projects were originally designed by the same person, Pieter Hintjens.

He was involved in the AMQP mess, not in designing RabbitMQ.

At least if you took his blogs at face value, he wasn't involved in the "mess", but the mess was forced on everyone by corporate interests.

And I believe this to be true. Peter was a brilliant and wonderful human being whom I shared a tremendous amount in common and largely the same viewpoint on life with. There's not a week that goes by where I don't mourn his loss and where we could be today if he were still with us.

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#177

Post by benterix »

xnickb wrote: post #144
sofixa wrote: post #140

Earlier quoted context omitted.

> And then there is Helm Right, who doesn't want to template hundreds of lines of code in a language that uses whitespace for logic and was never made neither for templating nor complex long documents(YAML)? What could possibly go wrong ("error missing xxx at line 728, but it might be a problem elsewhere").

But my infrastructure is code! Can't you see how it's all in git?

What's wrong with having your infrastructure as code and storing it in Git?

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#178

Post by benterix »

keredson wrote: post #122
nosefrog wrote: post #105

Earlier quoted context omitted.

To be fair, I worked on multiple projects removing queues at Google, so it's more than just that.

and mandates that virtually all new projects not directly use borg/kubernetes.

That's interesting. What's the rationale behind that?

Re: Ask HN: Why do message queue-based architectures seem less popular now?

#180

Post by alfiedotwtf »

ithkuil wrote: post #156
hn_throwaway_99 wrote: post #102

I like a lot of the answers, but something else I'd add: lots of "popular" architectures from the late 00s and early 2010s have fallen by the wayside because people realized "You're not Google. Your company will never be Google." That is, there was a big desire around that time period to "build it how the big successful companies built it." But since then, a lot of us have realized that complexity isn't necessary for…

We also have much much bigger single machines available for reasonable money. So a lot of reasonable workloads can fit in one machine now that used to require a small cluster

This! You NEEDED to scale horizontally because machines were just doing too much. I remember when our Apache boxes couldn’t even cope doing SSL so we had a hardware box doing it on ingress!
Post reply on HN