Why our team cancelled our move to microservices • Hacker News Board

Re: Why our team cancelled our move to microservices

#231

Post by whimsicalism » Wed, Jan 26, 2022, 8:30 PM UTC

BackBlast wrote: post #227

whimsicalism wrote: post #62

Earlier quoted context omitted.

> What have you gained? Separation of responsibilities? Easier to analyze because you only have so many inputs and outputs to a simpler system? Debugging something that touches a lot of paths in a monolith can be quite nightmarish as well.

If I'm in Java, JavaScript, or Python and there is a code fault, the system provides me a stack trace of the call structure that lead to the error. If I catch the error I can output more related data as I deem necessary. This comes effectively, out of the box. How do I do a similar stack trace in microservices to understand the path that led to this state? I've used microservices at a couple of companies and their me…

> If I'm in Java, JavaScript, or Python and there is a code fault, the system provides me a stack trace of the call structure that lead to the error. If I catch the error I can output more related data as I deem necessary. This comes effectively, out of the box.

Sure, if you are not writing a program using event-driven async style. Every monolith I've worked with has been in async style pretty much.

With global state in the monolith, this can become quite difficult to reason about. By contrast, with microservices, you can analyze the service as performing a small function with a single input and output without global state dependencies. This can be easier to debug.

Re: Why our team cancelled our move to microservices

#232

Post by midrus » Wed, Jan 26, 2022, 8:43 PM UTC

bhuber wrote: post #85

rydgel wrote: post #42

Why would you consider microservices if you are only 12 developers?

Most of the conversation so far has focused on the development benefits of microservices (decoupling deployments, less coordination between teams, etc). Small teams don't really have this problem, but there are other benefits to microservices. One of the biggest is scaling heterogeneous compute resources. Suppose, for example, your webapp backend has to do some very expensive ML GPU processing for 1% of your incoming…

Well, no. You can deploy the same monolith to two different clusters with different resource configurations.

In fact, this is what you usually do with "worker" nodes that do background jobs.

And you can always have feature flags/environment variables to disable everything you don't need in a given cluster.

Re: Why our team cancelled our move to microservices

#233

Post by midrus » Wed, Jan 26, 2022, 8:48 PM UTC

Microservices and SPAs are to me in the same category of "things we do because it is fun" rather than because the business will take any benefit out of it.

The amount of effort wasted is just not worth it in 90% of the cases.

Re: Why our team cancelled our move to microservices

#234

Post by bhuber » Wed, Jan 26, 2022, 10:31 PM UTC

midrus wrote: post #232

bhuber wrote: post #85

Earlier quoted context omitted.

Most of the conversation so far has focused on the development benefits of microservices (decoupling deployments, less coordination between teams, etc). Small teams don't really have this problem, but there are other benefits to microservices. One of the biggest is scaling heterogeneous compute resources. Suppose, for example, your webapp backend has to do some very expensive ML GPU processing for 1% of your incoming…

Well, no. You can deploy the same monolith to two different clusters with different resource configurations. In fact, this is what you usually do with "worker" nodes that do background jobs. And you can always have feature flags/environment variables to disable everything you don't need in a given cluster.

I'm not saying you have to use microservices to solve these problems, just that they are potential reasons why you might want to, even with a small team of developers. I would also argue that if you're deploying the same codebase in two different places and having it execute completely different code paths, you're effectively running two separate services. Whether or not you decide to deploy a bunch of dead code (i.e. the parts of your monolith that belong to the "service" running on the other cluster) along with them doesn't change how they logically interact.

Re: Why our team cancelled our move to microservices

#235

Post by BackBlast » Wed, Jan 26, 2022, 11:51 PM UTC

whimsicalism wrote: post #231

BackBlast wrote: post #227

Earlier quoted context omitted.

If I'm in Java, JavaScript, or Python and there is a code fault, the system provides me a stack trace of the call structure that lead to the error. If I catch the error I can output more related data as I deem necessary. This comes effectively, out of the box. How do I do a similar stack trace in microservices to understand the path that led to this state? I've used microservices at a couple of companies and their me…

> If I'm in Java, JavaScript, or Python and there is a code fault, the system provides me a stack trace of the call structure that lead to the error. If I catch the error I can output more related data as I deem necessary. This comes effectively, out of the box. Sure, if you are not writing a program using event-driven async style. Every monolith I've worked with has been in async style pretty much. With global state…

It seems to me that you are comparing apples and oranges. If your code has no relevant state or coupling, a pure function. Sure, you can look at it in isolation. You could do the same in a monolith.

I've worked on plenty of monoliths with async style code and have found the stack traces to be plenty helpful. It's never been an issue. At least in JavaScript you retain all relevant scoped data and can dump it all if you like. Debugging gets much more interesting when there are no obvious such errors, yet error conditions are present.

I'm still not seeing any equivalent microservice tooling.

Re: Why our team cancelled our move to microservices

#236

Post by tomc1985 » Thu, Jan 27, 2022, 12:27 AM UTC

dpark wrote: post #221

tomc1985 wrote: post #210

Earlier quoted context omitted.

> Marketing teams know that as well, and they exploit that as much as they can. They exploit this mercilessly , just as hard as they exploit our curiosity and desire for novelty as engineers. To the GP, your company's marketing/sales team may not drive internal technical decisions, but I can promise you that other companies' sales teams do . Why do you think they so persistently step on the heads up their first POC s…

This is very “everyone else is such an idiot”. I feel like you’re a half step away from saying that engineers shouldn’t be allowed to learn about new tech because they’re too stupid to know when not to apply it. If your team of engineers is incapable of determining good engineering trade-offs, and you have too little influence on the team to steer them back to a reasonable choice, it’s not really marketing’s fault. Y…

Not at all. I am simply describing my experience with salescritters as a SWE. A good company gives Engineering a seat at the table but sometimes you have to deal with things imposed from higher up. This is especially true in a large company.

That said I think that we are a particularly gullible cohort with misplaced desire for novelty and shiny tech. We overrationalize in favor of these desires and have a hard time walking a mile in others' shoes. And companies know this and exploit this to constantly fill our heads with noise and bad ideas that happen to be profitable for them. This is one of my biggest beefs with "tech" in general, and why I am taking some time off from this trade.

Re: Why our team cancelled our move to microservices

#237

Post by randerson » Thu, Jan 27, 2022, 1:31 AM UTC

scottious wrote: post #163

giaour wrote: post #31

> Microservices allow your team to have control over the full stack they require to deliver a feature. This is honestly pretty rare, at least in my experience. What I have seen is that organizations will buy in to the microservices hype, then dictate to their teams what stacks, deployment paradigms, etc. (sometimes even down to the sprint cadence) are acceptable.

or even worse: the teams DO have full control over their stack and now you have dozens or more different technology stacks at the company and expertise isn't shared between them

Extra fun when you're on-call for one of those stacks that only 2 people in the company have experience with, and you can't get hold of either of them.

Re: Why our team cancelled our move to microservices

#238

Post by wussboy » Thu, Jan 27, 2022, 1:59 AM UTC

QuercusMax wrote: post #173

candiddevmike wrote: post #122

Earlier quoted context omitted.

Instead of shuffling data between things via shared memory/function args, you're doing it over the network. This will always be more complicated.

Here at Google, one of our most popular microservices frameworks enables microservices to be assembled into servers. If one microservice calls another one in the same assembly, it won't touch the network, and is quite optimized. There's no reason that microservices have to all run in separate binaries, but when it's useful it can be done easily enough without having to change your code.

I find my self doing the opposite. I’ll often create a Cloud Task that will call the same server that created it, just to make sure that another container will get spun up to handle the load if needed. Using Cloud Run this way quite happily for a mix of OLAP and OTAP workloads.

Re: Why our team cancelled our move to microservices

#239

Post by joatmon-snoo » Thu, Jan 27, 2022, 4:24 AM UTC

jedberg wrote: post #66

joatmon-snoo wrote: post #36

Monolith vs microservice is not a dichotomy, it's a spectrum (as it is with so many other things). Individual microservices can still become monolith-y and become responsible for doing a Lot of Things (TM). That being said, the biggest hurdle in a re-architecture project like this is usually in the "n=1 -> n=2" stage, and "n=2 -> n=5" is a lot easier: once you add service #2, you learn how to set up telemetry, permis…

I always say n=2 -> n=3 is the hardest step, then n=1 -> n=2. With 1 to 2, you can take a lot of shortcuts with the n's communicating directly. With 3, you have to start formalizing message transmission either through a message bus or a cache or whatever. But once you have n=3, n=3+ is pretty easy as it's mostly edge cases to solve for or geometric scaling problems.

It definitely depends on the shortcuts you take! e.g. if you run #1 and #2 with the same identity, you don't have to figure out how to make your ACLs work... but you're also breaking principle of least access :)

Re: Why our team cancelled our move to microservices

#240

Post by dagss » Thu, Jan 27, 2022, 6:49 AM UTC

pc86 wrote: post #183

dagss wrote: post #52

Earlier quoted context omitted.

"just picked a bad service boundary" -- well that's the thing isn't it. If you always pick the right boundary up front, something that is perfect both now and also anticipates any kind of future crazy feature request -- if you can pull that off I'd say any architecture will work well. But most people get boundaries wrong some times. Sometimes very badly wrong. Sometimes the boundaries are historical, set by product o…

Isn't the point of making services as small as possible so that you can easily shift boundaries? Isn't DDD (a common companion of microservices) all about constantly shifting business domains? We've gotten boundaries wrong tons of times. We change them, which includes a migration script to move historical data from one service to another, if possible. Yes, it's work, but it's not any more work than having everything…

Well the parent talked about not joining across services. Then there are hard limits to how small you can make them.

And what need to join with what is something that can change.

So if you make services very small, and no joins across services, you get a lot of copies of the same data everywhere...

And if you make the services too small you have just moved the exact patterns you would have inside a monolith to into your APIs..how can that be easier?

You say "downsides of monoliths" but monoliths isn't a homogenous thing, they come in all shapes and varieties. So does microservices.

Myself I happen to have experienced microservices systems that are a real mess, and pretty clean monolith designs. Consider for instance an event sourced service where every endpoint a) reads events from database, b) if business rules allows (involving possibly external calls) writes an event to the database. No CRUD. This pattern keeps every handler/business rule reasonably isolated from other, and it doesn't matter if it runs in 1 or 10 services...