Meta’s Microservice Architecture [pdf] • Hacker News Board

Re: Meta’s Microservice Architecture [pdf]

#101

Post by charcircuit » Tue, Jul 11, 2023, 4:50 PM UTC

RicDan wrote: post #94

charcircuit wrote: post #83

Earlier quoted context omitted.

You run another instance of it.

So you have to take care of routing etc. I see how it works, and I completely agree that to start out, so going from PoC to first business implementation, a monolith is the way to go (unless the goal from the start is 100 million concurrent users I guess). But after that initial phase, does it really matter if you use one or the other? You can overengineer both and make them a timesink, or you can keep both simple. I…

>So you have to take care of routing etc.

The routing of just load balancing is much simpler than the routing of exectution jumping between many microservices.

>You can overengineer both and make them a timesink

I agree, but a microservice architecture starts you out at a higher complexity.

>but being able to completely isolate business logic seems like a nice gain

That can also be done by having that business logic live in its own library.

Re: Meta’s Microservice Architecture [pdf]

#102

Post by steveBK123 » Tue, Jul 11, 2023, 4:51 PM UTC

steveBK123 wrote: post #67

The trick is to use a slow enough language so no one notices the IPC hit of micro services.

Man the quickest way to get downvoted here is to dunk on Python huh?

Re: Meta’s Microservice Architecture [pdf]

#103

Post by dboreham » Tue, Jul 11, 2023, 4:54 PM UTC

AndrewPGameDev wrote: post #36

I've had this thought for a long time that if you have a completely functional code base (as in no side-effects), making the decision between a microservice approach and a monolith approach is fundamentally transparent. Any functional code can not only be split onto multiple threads, but multiple servers (at the cost of latency).

> Any functional code can not only be split onto multiple threads, but multiple servers (at the cost of latency).

Assuming some simplifying assumptions, yes. But those simplifying assumptions tend not to hold in the real world.

Re: Meta’s Microservice Architecture [pdf]

#104

Post by lightbendover » Tue, Jul 11, 2023, 4:57 PM UTC

jameshart wrote: post #74

fvdessen wrote: post #35

Earlier quoted context omitted.

That also works with monoliths ... Usually you would make your monolith stateless and distribute the incoming requests / events across many instances that can be spawned / killed depending on volume of requests and health status of instances.

When you kill a monolith you kill a random selection of inflight tasks from every part of your application. So a rare bug in your mailing list signup workflow that hangs the process and causes it to be killed causes a random selection of inflight webpage requests, payment transactions, message handlers and business processes to fail. And if those failures aren’t all cleanly handled, your mailing list signup bug could…

Generally when people argue in favor of a monolith over micro-services, it's not for completely/mostly isolated business functions (i.e. BI pipelines vs. CMS CRUD), it's more for when responding to a single request of group of requests already implicates many services that must all work together to respond to the request(s); in this case, you're still smoked if any one of those services handling a part of the request chokes, you're in fact multiplying your opportunities for failure if you're using micro-services.

Monoliths should be stateless (if achievable) and have no concept of partial success in cases where you would like atomicity unless everything is truly idempotent (easier said than achieved). If those criteria are met then callers just need to retry in the event of failure which can be set up for basically free in most frameworks.

If you're pushing fatal recurring bugs into production, then that is a separate problem wider than the scope of a monolith vs. micro.

Re: Meta’s Microservice Architecture [pdf]

#105

Post by shagmin » Tue, Jul 11, 2023, 5:04 PM UTC

treis wrote: post #65

philjohn wrote: post #59

Earlier quoted context omitted.

In a past job, the benefit of microservices was that some of the operations performed by the system were fare more CPU intensive than others - by having them in their own service that could be scaled independently led to lower overall hardware requirements, and made keeping latency of the other services sensible much easier.

You can scale monoliths independently too. Depending on the language that means paying some additional memory overhead for unused code but practically it's small compared to the typical amount of ram on a server these days.

This post reminds me of exactly the balance I've been toying with. One particular service I work with has ~6 main jobs it does that are all kind of related in some way but still distinct from each other. That could've been designed as 6 microservices, but there are services that do other things as well - it's not all contained in giant one monolith so it's somewhere in the middle.

The software is going to be deployed at different locations with different scaling concerns. In some places, it's fine to just run 1 instance where it does all 6 jobs continuously. At other places, I anticipate adding parameters or something so it can run multiple instances of a subset of the jobs, but not necessarily all the jobs on every instance.

Re: Meta’s Microservice Architecture [pdf]

#106

Post by joshuamorton » Tue, Jul 11, 2023, 5:19 PM UTC

mostlysimilar wrote: post #97

jameshart wrote: post #74

Earlier quoted context omitted.

When you kill a monolith you kill a random selection of inflight tasks from every part of your application. So a rare bug in your mailing list signup workflow that hangs the process and causes it to be killed causes a random selection of inflight webpage requests, payment transactions, message handlers and business processes to fail. And if those failures aren’t all cleanly handled, your mailing list signup bug could…

The original statement was "service is not answering for a certain amount of time". If the instance of your monolith is not responding you're probably already in a bad state and can reasonably kill it.

Sure, but "if the instance of your monolith is not responding" probably means the app is down. That's only going to be true for a small subset of the microservices.

Re: Meta’s Microservice Architecture [pdf]

#107

Post by totallywrong » Tue, Jul 11, 2023, 5:23 PM UTC

It's cool to hate on microservices here on HN. In my opinion it's not always the answer but can bring many benefits. For example the ability to use different languages or frameworks that might be better suited for parts of the application, the separation of concerns, the ability to split work between teams, outsourcing parts of the app, scaling parts of the app individually, etc. They key is to have clear API contracts for each service, and enforce them ruthlessly.

Re: Meta’s Microservice Architecture [pdf]

#108

Post by phyrex » Tue, Jul 11, 2023, 6:38 PM UTC

MarkSweep wrote: post #86

phyrex wrote: post #25

I find this highly misleading, Facebook is famously a big monolith, and I think so is instagram. There’s plenty of services and a couple micro services as well, but I don’t think anybody would characterize meta as having a micro services architecture. I have no idea what the authors’ agendas are, but something isn’t right there.

The article mentions that www, the monolithic PHP code base, is 4.6% of the service instances. Though the paper does not mention the compute allocation. I do not recall how www is deployed, but it’s easy to imagine it’s allocated a lot more resources than a lot of other services are.

> At Meta, 0.1% and 1% of the largest services consume 40% and 80% of the total fleet capacity, respectively.

from https://www.usenix.org/system/files/osdi23-grubic.pdf

Re: Meta’s Microservice Architecture [pdf]

#109

Post by 0xbadcafebee » Tue, Jul 11, 2023, 7:03 PM UTC

jerf wrote: post #93

0xbadcafebee wrote: post #63

Earlier quoted context omitted.

They can consider it all they want, that won't change the outcome. It is inevitable.

You can't avoid its impact, but you can either work with it or work against it. I assume that was more the point.

I don't think you can, actually, from the level of a team. If you're designing a bridge, you can work with gravity, or against it. But if you're a car on the bridge, you can't work with or against it; the bridge either will hold you up or it won't, your car will either go or it won't.

Conway's Law is about how your organization's communication structures work, and how that inevitably leads to specific outcomes. That outcome is an eventuality for the team, because the team can't change the communication structure. They can design the best damn microservice in the world, and the resulting use of it could still be shit, because the other teams aren't necessarily up to snuff. And you still have all the baggage of dealing with those intra-team, intra-service issues.

Re: Meta’s Microservice Architecture [pdf]

#110

Post by jameshart » Tue, Jul 11, 2023, 8:18 PM UTC

lightbendover wrote: post #104

jameshart wrote: post #74

Earlier quoted context omitted.

When you kill a monolith you kill a random selection of inflight tasks from every part of your application. So a rare bug in your mailing list signup workflow that hangs the process and causes it to be killed causes a random selection of inflight webpage requests, payment transactions, message handlers and business processes to fail. And if those failures aren’t all cleanly handled, your mailing list signup bug could…

Generally when people argue in favor of a monolith over micro-services, it's not for completely/mostly isolated business functions (i.e. BI pipelines vs. CMS CRUD), it's more for when responding to a single request of group of requests already implicates many services that must all work together to respond to the request(s); in this case, you're still smoked if any one of those services handling a part of the request…

If you have a guaranteed way to avoid pushing fatal bugs to production it doesn’t MATTER what your architecture is. You’re in some fantasy land of rainbows and kittens where everything works fine first time every time.

For those of us in the real world who can’t afford perfection, the ability to isolate the impact of the inevitable bugs that do sneak through has some appeal.

As does the fact that exhaustively testing a microservice in a realistic timeframe is a much more tractable problem than exhaustively testing a monolith, which reduces the risk that such bugs will ship in the first place.

Bugs are less likely to ship. And when they do they will have a more limited blast radius. And when they’re detected they can be mitigated more quickly.

Those all sound like great benefits to me.