Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Starting with microservices

arnoldgalovics.com

Post reply on HN
Print view View original

41–50 of 139 posts

Re: Starting with microservices

#41

Post by jiggawatts »

Architecture astronauts love microservices.

I'm watching a government customer take simple, cohesive systems developed by a small team (4-5 people) and split it up into tiny little pieces scattered across different hosting platforms and even clouds.

Why?

Because it's "fun" for the architects and pads out their resume.

Just now, I'm watching a "digital interactions" project that will have dozens of components across two clouds, including multiple ETL steps. In all seriousness, half of that could be replaced by a database index, and the other half with a trigger.

They're seriously going to deploy clusters of stuff on Kubernetes for 100 MB of data to make sure it "scales"... to 200 MB. Maybe. Eventually.

What kills me is that now that they've made the decision to over-engineer the thing, my consultancy firm can't bid on the tender because we don't have the appropriate experience building over-engineered monstrosities!

The simple and effective solutions to problems we've delivered in the past are disqualifying us from work.

You guessed it: my next project will be an architect's wet dream and will be over engineered just so that we can say on tender applications that "yes, we have the relevant experience".

Gotta play the game by the rules...

Re: Starting with microservices

#42

Post by bob1029 »

scanr wrote: post #10

I have a theory that auto-wired Dependency Injection in a single DI container is partly to blame for monolith spaghetti. Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult. I think there is value in wiring a monolith together in such a way that each course grained subcomponent exposes a constrained interface into the rest of the system (payments…

> Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult.

I have experienced this pain so many times and in so many different varieties. Often, you cannot meaningfully subdivide the problem space without ruining the logical semantics per the business (i.e. bowl of spaghetti).

An alternative is to embrace the reality that circular dependencies are actually inevitable and appropriate ways to model many things.

The example scenario I like to use is that of a typical banking customer. One customer likely has a checking and savings account. For each of those accounts, there are potentially multiple customers (joint ownership). Neither of these logical business types will ever "win" in the real world DI graph. Certainly you can start to invent bullshit like AccountCustomer and CustomerAccount, but that only gets you 1 layer deeper into an infinitely-recursive rabbit hole of pain and suffering. There also exists the relational path, but I have heavily advocated for that elsewhere and it is not always applicable when talking about code-time BL method implementation. Being able to model things just as they are in the real world is a big key to success in the more complicated problem domains.

Instead of trying to control what depends on what, I shifted my thinking to:

> What needs to start up and in what order?

Turns out, most things don't really care in practice. The only thing I have to explicitly initialize before everything else in my current code base is my domain model working set (recovery from snapshot/event logs) and settings. I decided to not use DI for any business services. Instead, all services become a static class with static members that can be invoked from anywhere. This also includes the domain model instance which is used as the in-memory working set. This type just contains an array of every subordinate domain type (Customers, Accounts, etc.). By having the working set available as a public static type, every service can directly access it without requiring method injection. If I was working with a different problem domain (or certain bounded context within this one), I might prefer method-level injection.

Yes - according to every book on programming style you ever read, this is an abominable practice. Unit testing this would be difficult/impossible. But you know what? It works. It's simple. I can teach a novice how to add a new service in an hour. A project manager stumbling into AccountService might actually walk away enlightened. You can circularly-reference things at runtime if you need to. I've got some call graphs that bounce back and forth between customer & account services 5+ times. And it totally makes sense to model it that way too as far as the business is concerned. Everyone is happy.

Re: Starting with microservices

#43

Post by frozenport »

jayd16 wrote: post #23
bob1029 wrote: post #19

I have done the full theme park ride on monolith->microservices->monolith. Both have ups and downs. The most important thing I learned is that "microservices" in absolutely no way necessitates "bullshit spread across multiple cloud vendors and other scenarios involving more than 1 computer". What part of microservices says things must be separated by way of an arbitrary wire protocol? We now have a "monolith" (proces…

Farming out libraries or subprojects to separate teams is an ok way to go as long as you don't mind a single language/build system. Its not as flexible but you maybe get some perf out of it. You don't get the convenience of a single datastore with simple transactions but it sounds like you prefer the flexibility of services owning their own data. As it turns out there's no silver bullet. Do what works for you.

>> single language/build system

Imagine designing your entire architecture around your build system.

Re: Starting with microservices

#44

Post by marginalia_nu »

Lifecycle management is also a big part of it, I think.

My search engine is a hybrid architecture, with some monoliths and some microservices.

* Index - 5+ minute start time and uses 60% of system RAM.;

* Search Server (query parser, etc.) - 30 second start time due to having to load a bunch of term frequency data, low resources, ephemeral state

* Assistant Server (dictionary lookups, spell checks, etc.) - fast start, medium resources, stateless

* Crawler - only runs sometimes, high resources, statefull

* Archive Server- fast start, low resources, ephemeral state

* Public API gateway - 5 second start time, low resources, ephemeral state

+ a few others

A lot of the ways it's divided is along the lines of minimizing disruption when deploying changes. I don't have the server resources to run secondary instances of any of these, so I'm working with what I've got. If I patch the crawler, I don't want the search function to go down. If I patch the search function, I don't want to have to wait 5 minutes to restart the index.

It would certainly be cleaner to say break apart the Index service in terms of design, as it does several disparate things, but those things have a resource synergy which means I can't, not without buying another server and running a small 100 GbE network between them. Seems silly for a living room operation.

Re: Starting with microservices

#45

Post by teekert »

We have a big codebase where I work and it’s written as a set of micro services. I’m a bit of a noob but I thought that micro services were always containerized. Turns out we don’t do that and it all runs in a very bespoke and poorly documented VM. Now can we call this a microservices based architecture? I feel like we cannot, it feels like worst of both worlds…

Re: Starting with microservices

#47

Post by bob1029 »

brundolf wrote: post #34
bob1029 wrote: post #19

I have done the full theme park ride on monolith->microservices->monolith. Both have ups and downs. The most important thing I learned is that "microservices" in absolutely no way necessitates "bullshit spread across multiple cloud vendors and other scenarios involving more than 1 computer". What part of microservices says things must be separated by way of an arbitrary wire protocol? We now have a "monolith" (proces…

You could even have a mono-repo that holds services which run as separate processes, if you need that. That way you still get the benefits of sharing code, types, version numbers, build processes, etc, which seem like the main headaches with the usual approach.

We passed right by this mode of operation while migrating from microservices=>monolith. Everything is a separate process but running on the same box. Ran that way for quite some time. We pulled services into the main process one-by-one. It went about as smoothly as one could have hoped for.

Re: Starting with microservices

#48

Post by 4rb1t »

scanr wrote: post #10

I have a theory that auto-wired Dependency Injection in a single DI container is partly to blame for monolith spaghetti. Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult. I think there is value in wiring a monolith together in such a way that each course grained subcomponent exposes a constrained interface into the rest of the system (payments…

In theory it all sounds great and makes total sense but when the company grows to a billion dollar firm and hires engineers to work on the said monolith thats when things breakdown. The founding team was a closet knit team and ensure there is no spaghetti code but the moment you move on from that closely knit group its hard to enforce constraints.

Have worked at multiple companies that started out as a monolith and are still running the monolith in some form or the other while breaking it down into micro services.

Re: Starting with microservices

#49

Post by throwaway894345 »

scanr wrote: post #10

I have a theory that auto-wired Dependency Injection in a single DI container is partly to blame for monolith spaghetti. Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult. I think there is value in wiring a monolith together in such a way that each course grained subcomponent exposes a constrained interface into the rest of the system (payments…

I’ve never been part of a monolith that used a DI framework, but I’ve seen quite a few monoliths fail (as in “the project becomes too convoluted and iteration slows to a crawl until the project is canceled or effectively rewritten”) and I certainly believe that one important reason microservices do well is that they enforce the modularity that you describe. That said, a lot of critics of microservices describe similar issues of indiscernible chaos, so either I’ve been very fortunate or microservice critics are gaslighting us. :)
Post reply on HN