Live data from Hacker News

It's not microservice or monolith; it's cognitive load

fernandovillalba.substack.com

51–60 of 195 posts

Re: It's not microservice or monolith; it's cognitive load

#51
post #44
post #30

Earlier quoted context omitted.

This strikes me as overly cynical. My current job is working with around 20 other engineers on an extremely bloated and coupled monolith. I'd love to be able to separate myself and my team from others by an agreed-upon interface. Yes, "agreed-upon" is certainly doing some heavy lifting there, but I don't think it's realistic to expect that > cross team meetings and project management every time you want to ship a fea…

> I'd love to be able to separate myself and my team from others by an agreed-upon interface. Don't doubt it, but that is what the grandparent was going to - you don't want to be held accountable for the success of the overall product, you want a defined contract that you maintain and the value-add of that is someone else's problem. The dysfunction of a business is made up of individuals doing what makes the most sen…

Isn't this precisely what leadership is for? If you've implemented an architecture that mirrors your org chart, then the interfaces aren't just in software, they exist between managers as well. System-level success depends on the managers and the directors above them to steer the boat in a way that places accountability on the teams and components of the architecture necessary to ship a feature, or whatever.

Re: It's not microservice or monolith; it's cognitive load

#52
post #22

Earlier quoted context omitted.

This is wrong. That's not how Conway's Law works. It’s also not how software design works. But please, by all means, continue to spread disinformation and keep us in the dark ages.

I don't disagree completely with the comment, but I would possibly change it from "comply with Conway" tô "embrace Conway".

If you don't disagree, then I assume you have a strong understanding of structural design. I assume that you recognize that Conway's law is more of a curse, and a warning than it is something to embrace. I assume that you recognize that the only possible way to "embrace conway's law" and simultaneously recognize structural design would be to constantly be firing or otherwise disbanding entire teams as components get completed (components that likely won't need to be touched frequently because they were designed for a single reason to change). I assume all of that makes perfect sense to you. Yes?

Re: It's not microservice or monolith; it's cognitive load

#53
post #30

Earlier quoted context omitted.

This strikes me as overly cynical. My current job is working with around 20 other engineers on an extremely bloated and coupled monolith. I'd love to be able to separate myself and my team from others by an agreed-upon interface. Yes, "agreed-upon" is certainly doing some heavy lifting there, but I don't think it's realistic to expect that > cross team meetings and project management every time you want to ship a fea…

> This strikes me as overly cynical. > My current job is working with around 20 other engineers on an extremely bloated and coupled monolith. I'd love to be able to separate myself and my team from others by an agreed-upon interface. > Yes, "agreed-upon" is certainly doing some heavy lifting there, but I don't think it's realistic to expect that This is overly cynical to you likely because you haven't experienced mai…

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run).

You have experienced what happens when web developers cosplay as software architects. That's when you get "microservices". It's a travesty, and it has ruined an entire generation of software developers who now believe that partitioning and autonomy are just myths.

So, as a blunt statement - your experience is typical. That is, it's average. That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.

Re: It's not microservice or monolith; it's cognitive load

#54
post #53

Earlier quoted context omitted.

> This strikes me as overly cynical. > My current job is working with around 20 other engineers on an extremely bloated and coupled monolith. I'd love to be able to separate myself and my team from others by an agreed-upon interface. > Yes, "agreed-upon" is certainly doing some heavy lifting there, but I don't think it's realistic to expect that This is overly cynical to you likely because you haven't experienced mai…

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run). You have experienced what happens when web developers cosplay as software architects. That's when you get "microserv…

Just to clarify, what are components referring to? Are they independent web services?

And what is the total size of the developers you worked with? Are the 12 just your team or the entire engineering org?

Edit: found from another comment

> We don’t call them microservices, because we aren’t web developers pretending to do architecture and recreating the mistakes of DCOM, CORBA, and Web Services. Most of them are autonomous event sourced components. The others are stand alone web applications that are stitched together with Nginx routing and SSI.

Re: It's not microservice or monolith; it's cognitive load

#55
Lets step back a second. If the rationale behind adopting microservices for everything is PURE orgitecture rather than software architecture, then that's not really a rationale at all.

Instead of having one instance, you now have dozens of little service fiefdoms plus all the added network I/O overhead associated with that.

The principled approach is really to simply not do that, for the basic latency costs. I mean, did World of Warcraft, which is way more impressive than 99% of all the little django apps out there in corpo world run on a gajillion microservices? Fuck no

Re: It's not microservice or monolith; it's cognitive load

#56
post #53

Earlier quoted context omitted.

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run). You have experienced what happens when web developers cosplay as software architects. That's when you get "microserv…

Just to clarify, what are components referring to? Are they independent web services? And what is the total size of the developers you worked with? Are the 12 just your team or the entire engineering org? Edit: found from another comment > We don’t call them microservices, because we aren’t web developers pretending to do architecture and recreating the mistakes of DCOM, CORBA, and Web Services. Most of them are auto…

Components are either autonomous event sourced back end components (using Eventide) or independent web applications that are combined with Nginx routes and SSI. The UI is (mostly) server-rendered Rails. The users have no idea they are hitting 20 or so different web applications, but the developers sure feel the productivity boost of every application being a small application that is (relatively) independent from one another.

A dozen or so is the entire software development organization.

Edit: Indeed! If you have any other questions, ask away. Most people think what we do isn't actually possible, but it is.

Re: It's not microservice or monolith; it's cognitive load

#57

Lets step back a second. If the rationale behind adopting microservices for everything is PURE orgitecture rather than software architecture, then that's not really a rationale at all. Instead of having one instance, you now have dozens of little service fiefdoms plus all the added network I/O overhead associated with that. The principled approach is really to simply not do that, for the basic latency costs. I mean,…

This is what you get when all you talk about it the positives, especially to new programmers or students. It's not about the why, but only if you do or do not.

I'll die on this hill, but DDD can be put right up there with microservices as a solution to a certain type of problem being sold as THE solution.

Re: It's not microservice or monolith; it's cognitive load

#58
post #53

Earlier quoted context omitted.

> This strikes me as overly cynical. > My current job is working with around 20 other engineers on an extremely bloated and coupled monolith. I'd love to be able to separate myself and my team from others by an agreed-upon interface. > Yes, "agreed-upon" is certainly doing some heavy lifting there, but I don't think it's realistic to expect that This is overly cynical to you likely because you haven't experienced mai…

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run). You have experienced what happens when web developers cosplay as software architects. That's when you get "microserv…

> So, as a blunt statement - your experience is typical. That is, it's average.

Definitely a fair judgement and would agree that this constitutes poor architecting on many many companies' parts including ones that I've been a part of.

> Every one of our 100+ components maintained by our dozen person team can be run and tested on its own.

Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team?

More directly, what's wrong with a monolith at the dozen person team size?

Re: It's not microservice or monolith; it's cognitive load

#59

The moment you adopt service based teams with service based managers, say goodbye to engineers caring about working product. Say hello to cross team meetings and project management every time you want to ship a feature. It's pure vanity for a startup to think they will become the next AWS by adopting hard service-based contracts between teams.

This viewpoint doesn't translate to reality in my experience. Monolith or not, engineers care about working product if they are incentivized to care about working product (and if they aren't, they don't).

Code is code. Cognitive load is cognitive load. Doesn't matter how you organize it. Unless your company is still very small and simple, there's no single team that is going to be able to understand how the entire system works (and take ownership of every part of it working properly). I've worked in many monoliths where you still had to have "cross team meetings and project management every time you want to ship a feature".

Re: It's not microservice or monolith; it's cognitive load

#60
post #53

Earlier quoted context omitted.

If you have to "get all 15+ services running on your laptop", you've already lost. Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. The only reason we have Docker on our machine is to test the images we deploy (just to make sure they build and run). You have experienced what happens when web developers cosplay as software architects. That's when you get "microserv…

> So, as a blunt statement - your experience is typical. That is, it's average. Definitely a fair judgement and would agree that this constitutes poor architecting on many many companies' parts including ones that I've been a part of. > Every one of our 100+ components maintained by our dozen person team can be run and tested on its own. Can you elaborate on this? What is the primary benefit of running 100-300 compon…

> Can you elaborate on this? What is the primary benefit of running 100-300 components vs a monolith specifically with a 12 person team?

> More directly, what's wrong with a monolith at the dozen person team size?

Great question. It's honestly a hard question to answer because the real answer requires tacit knowledge, but I'll try anyway because you asked. I'll link a couple articles of longer form writing that may help to answer as well.

The benefit ultimately is productivity. That productivity comes from autonomy, which comes from partitioning.

Each component can be worked, tested, and deployed on its own. We never need to spin up more than one at a time, which means any time we are working on our code, we are typically working on 1/400th of it, and we know we are because we have 400 separate GitHub repositories. We can't accidentally get peanut butter in our mayonaise. Each part is independently built and tested.

When it's time to upgrade Rails, we can upgrade one application. This is 1/20th of our Rails code. That's a smaller batch. We know from basic flow principles that (see: Principles of Product Development Flow, Lean, etc.) that small batches are better for productivity.

We rarely have merge conflicts. We don't do pull requests. We branch when we need to. We are often working on a brand new, never been deployed project. We don't have our tests running in CI. Our tests run in less than a second in most projects on our laptops. With those that have UI, our tests typically run in less than 30 seconds which include thorough Capybara (UI Interaction) testing.

When we are working in any application, we only need to consider its direct efferents when making a change to any interfaces it exposes. We can trace those changes through easily when we need to make them. Because we practice software design, we don't often need to make large sweeping changes, but when we do, we can do them methodically without disruption.

The "worst" part is that some things are tedious. But we can automate tedious things. We can invent around tedious things. We used to have to deploy 15 web applications any time we had a style change. Now we deploy one application that supplies the CSS for all our applications via SSI (server-side-include). If we need to deploy 70 back end components, we can script that (with manual verification at each step). We can even spread that out across a couple team members and do it as an ensemble. It takes about an hour to deploy everything. We maybe do this once a month or so (when something highly afferent changes -- and guess what, we structure our design so that afferent things change as infrequently as possible).

Most things we work on we work until they are "complete". That is, they rarely change after that. We have components that have been in production for 3 years and haven't been substantially touched aside from Ruby upgrades and the like. They are still perfectly in control.

The list goes on and on.

Here are a few articles, feel free to poke around the others and ask any follow up questions.

Partitions and Compositions: https://github.com/aaronjensen/software-development/blob/mas...

The Mythical Monolith: https://github.com/aaronjensen/software-development/blob/mas...

Post reply on HN