Live data from Hacker News

It's not microservice or monolith; it's cognitive load

fernandovillalba.substack.com

161–170 of 195 posts

Re: It's not microservice or monolith; it's cognitive load

#161
post #153

Earlier quoted context omitted.

> Someone got to say it so I will. Most of your comments comes across like you've gone of the rails and started replacing competency, not to mention humility, with narcissism. Being so cocksure and backing it up with statements that you need to be as brilliant as you to "understand" is such a laughingly large red flag that I'm not surprised people interpret it as satire. Where did I say that, exactly? I’ve said nothi…

> into ad hominem and straw man attacks. That’s one of the worst part about this industry: people’s inability to debate and discuss Let's see here: > You have experienced what happens when web developers cosplay as software architects. > That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the b…

> Let's see here:

Yes, let's. I have no problem with push back.

> You have experienced what happens when web developers cosplay as software architects.

I agree that this is overly snippy to the point of being counter-productive. It represents a particular emotional frustration with the state of our industry. We knew what ended up being microservices (web API servers making web API calls to other web API servers) was a failure mode. We knew you couldn't just introduce network hops and call it "architecture". So yes, I'm annoyed about that such that I'm willing to call out nameless "web developers", but I understand how that can create discomfort.

> That is, you experienced an average team doing average work misguided by average bad advice. It says absolutely nothing about software design. It only speaks to the fat part of the bell curve doing what it always does.

Indeed, and the person I was responding to effectively acknowledged this. The sooner we can recognize that the skill curve/technology adoption curve are real things, and crossing the chasm takes hard work, the sooner we can stop leaving people behind said chasm. It's not easy. Nothing I said here is untrue. If it is, please point it out.

> because, unlike them, I have significant material experience in both camps.

Again, I'm pointing to a material difference. Most people who are attempting to refute what I am saying have never done what I am saying. They are doing it from a position of fear, uncertainty, and doubt, or worse. I've been where they are. I've fought the fight they are fighting and, thankfully, I lost, and was introduced to new ways of doing things and seeing things. If a person has done what I'm discussing, I would expect them to say that and tell me why it failed. Instead I get people telling me it can't work and I'm incompetent, etc.

> It’s hard to imagine worse advice. > Just read the author’s bio. This is a person that appears to have zero software design experience writing an article telling you to ignore software design and just respect your team configuration. I call this Conway’s Confusion.

Putting these together because they are about the OP article and the OP. "hard to imagine worse advice" is hyperbole, but it is bad advice. You can't force fit concepts to teams. I mean, you can try, but you'll always end up with unnaturalness.

The actual OP's bio talks about their career. None of it mentions software design. If a person with zero surgery experience started writing about how to properly set up operating rooms, you can darn well bet they will be called a charlatan and called to the carpet. If they aren't people may die. Software isn't that serious usually, but it's not hard to imagine that there have been billions upon billions of dollars flushed down the toilet for the sake of poor software design that no one speaks out against.

> But please, by all means, continue to spread disinformation and keep us in the dark ages.

Yes, overly cynical and unnecessary. I made my point prior to this and I didn't need to add this.

> That sets a good friendly tone huh? If you can't take push back, don't be an arse to begin with.

Some was unnecessary yes, thanks for calling it out. The rest represents what I think is healthy push-back against an orthodoxy that causes more harm than good. We can achieve more as an industry and make our way out of the realm of hobbyists and into something more serious. Many of us call ourselves engineers, but nothing we do resembles anything that people who are actually classically trained engineers do.

> > And which part, specifically sounds incompetent? I’m happy to discuss that.

> Your arrogance precludes a fruitful discussion. But I believe that this needs to be called out, if nothing else to nudge other people to also do it when they see it, or to, albeit much less likely, nudge you towards eating some humble pie.

Just to be clear: you said that what I was saying sounded incompetent. I asked you about that, and you're telling me I appear to be too arrogant for you to tell me why I sound incompetent? Please, tell me what I said that sounds incompetent. Let's move past the ad hominem portion of the discussion now and talk about the actual substance. I'll do my best to refrain from hyperbole and unnecessary snark.

Re: It's not microservice or monolith; it's cognitive load

#162
post #76

Reading the responses/comments in here, a question arises... Are things really that black and white as people here paint them - with monoliths unavoidably and necessarily becoming a tangled mess of spaghetti, and micro-services being the singular, only way to achieve clean separation, autonomy and partition? With microservices inevitably being an organizational nightmare that leads to problems in inter-team coordinat…

Monorepo, separate modules is the way to go. I just don’t think it occurs to people.

But I think for compiled languages people see the build steps as particularly wasteful, both locally and in CI. They forget about the integration test surface area or think it’s something to be paid for later (tech debt) and reason that if they are changing one module why should they have to pay to compile and test the whole thing?

A good tooling or ops person can ameliorate a lot of that. There are a lot of poor patterns that people ignore when build units are small and incremental, that you cannot ignore when they get large. And then there are tools that can auto start tests, which reduces the latency in code-build-test quite a bit.

Re: It's not microservice or monolith; it's cognitive load

#163
post #138

Earlier quoted context omitted.

Do you know what does stand the test of time? Design principles that are based on the immutable laws of physics. There's a reason that I can quote Plato to support the notion of partitioning. There's a reason that the basic principles are used in the design of every single significant thing in the world. There's a reason Linux has as many modules as it has (for them, a single monorepo works, but multiple works too --…

Nobody is denying partitioning is helpful. It’s the implementation details of that partitioning that cause the debate. IMO using repos as a partition boundary is an abuse of git. And IMO invoking Plato to make a point is not helpful. Argumentum ad antiquitatem. This goes for the “30 years” comment as well: it’s entirely conceivable that they’ve been doing things inefficiently for 30 years. More likely that there is s…

> IMO using repos as a partition boundary is an abuse of git.

How so? Why don't we put our Rails applications in the Rails repository?

> And IMO invoking Plato to make a point is not helpful. Argumentum ad antiquitatem

What I am pointing to is a common thread. Plato, Christopher Alexander, Edwards Deming, etc. What I am arguing for is anything but common tradition. It's counter to what is current "common knowledge".

> This goes for the “30 years” comment as well: it’s entirely conceivable that they’ve been doing things inefficiently for 30 years.

Yes, entirely possible. And if we find a better way to do things, we will be first in line.

> More likely that there is such a huge sunk cost that they are heavily invested in justifying it.

What's your evidence of this?

Re: It's not microservice or monolith; it's cognitive load

#164

Earlier quoted context omitted.

This whole business has had few conceptual ideas that stand apart from fashion and stood the tests of time. A lot of people look down on C now for various reasons, but it's still extremely useful and going strong 50+ years later. Even though I'm older than C itself in the last year or two using it intensively, I found new insights in how to arrange things and what to avoid to make things more robust and maintainable.…

As someone who has worked in monoliths and microservices, monorepos and github galaxies, I think it would be better if we could converge on a monorepo managed by something like Bazel where you can just create a new build target that lists its dependencies, and can work directly with the dependency graph via its querying mechanism. Using repositories to encapsulate dependencies and build scripts I think is much too he…

> When I have to deal with that I usually wind up treating it like a monorepo anyways by creating automation to check out all repos and manage syncing them.

Yes! Absolutely. With one key difference: You can't commit to multiple repos at the same time. You can certainly have them all checked out locally and search across them all, and we provide scripts to do that. Example from the Eventide project: https://github.com/eventide-project/contributor-assets

This is an open source project. Our project is based on many of the ideas and techniques. They've held up for 7 years or so.

Re: It's not microservice or monolith; it's cognitive load

#165

Earlier quoted context omitted.

Perhaps it is easier to understand when you remember that service is not a technical term? People provide service. In the macro economy, service comes from other companies. If you integrate an LLM into your application, you may use the services of OpenAI. If you integrate payment processing into your application, you may use the services of Stripe. Microservices are just like services, except offered within a micro e…

With no or minimal communication how is it possible to build anything new? In real world if you need to build something new and it requires 10 different companies to do that, it would be virtually impossible.

Punctuated equilibrium.

A lot of customer value comes from companies that assemble ideas that have existed in isolation. Customers think they hate that, but the numbers say they are wrong.

Foods are an early example. Most are rearranging the same dozen ingredients in different quantities and orders of operation.

Vertical integration is not an all or nothing proposition. Companies can make pretty unique solutions by specializing a couple of pieces.

Re: It's not microservice or monolith; it's cognitive load

#166
post #104

Earlier quoted context omitted.

In most cases there's going to be substantial overlap between the functionality required for various business functions, and even in differentiated systems the ideal technical domains may not map cleanly to business domains. It's very easy for this to result in multiple teams having half-assed, semi-compatible implementations of the same functionality. Some places try to address this by creating dedicated "core" or "…

>Unless you can truly, completely decouple systems -- making them separate products with dedicated teams -- you run a high risk of having a codebase that is more complex that people still need to understand in its entirety to work effectively. The worst of both worlds. Exactly what I meant. And to make it happen you need to understand underlying business domain. Separate product doesn't have to be huge.

For a lot of tech startups the underlying business domain can be nebulous and change significantly over time, and then you end up with substantial organizational debt. Well-established firms can get away with (and benefit from) more differentiation and formal structure because they tend to be able to make longer-term plans.

Re: It's not microservice or monolith; it's cognitive load

#167
post #57

Lets step back a second. If the rationale behind adopting microservices for everything is PURE orgitecture rather than software architecture, then that's not really a rationale at all. Instead of having one instance, you now have dozens of little service fiefdoms plus all the added network I/O overhead associated with that. The principled approach is really to simply not do that, for the basic latency costs. I mean,…

This is what you get when all you talk about it the positives, especially to new programmers or students. It's not about the why, but only if you do or do not. I'll die on this hill, but DDD can be put right up there with microservices as a solution to a certain type of problem being sold as THE solution.

I'm interested to hear more about your views on DDD - especially if you have examples where DDD has been actively harmful.

Usually my advice for anyone thinking about building a new piece of software for a particular business goal is to a) run an event storming workshop with a group of domain experts to help get a really good idea of events, actors, commands, information flow and clusters of behaviour, and then b) run a second pass where you think the domain through in terms of transactional boundaries and DDD aggregates, and then c) do a third pass where you think specifically about security constraints and how they can be met.

It's an expensive exercise in terms of time taken up front, but having completed it, hopefully the team have gained enough of an insightful understanding of the domain that they won't make silly hard-to-reverse mistakes like needing transactions that span service boundaries, or building demonstrably distinct domains which share similar concepts into uber objects spanning those domains, or having one service depend on information from multiple other services in order to apply required security constraints.

Anyway, TLDR is I have found DDD and DDD-adjacent methods extremely helpful for thinking through designs and making app architecture decisions.

Re: It's not microservice or monolith; it's cognitive load

#168

Earlier quoted context omitted.

Depending on what exactly it is that you're storing or processing, neither does data. Think sensor data and the sensor is a vital signs monitor in a hospital. The service that reads its output and stores it glitches out due to some sort of misclick by a user. It distorts the fact that the patient has an arrhythmia. Or a service that reads off the medication dosages for a patient for a pharmacist is stuck on a single…

I strongly doubt they're debating microservice vs monolith in that area. Maybe you should try an example with cat photos.

Used to work for an EMR company. This is very much a real-world concern, I assure you.

Re: It's not microservice or monolith; it's cognitive load

#169

Earlier quoted context omitted.

It turns out that at a certain point, features start requiring integration in the system. Let's take a hypothetical SaaS that provides workflow management for a domain. Let's say there are about 125 engineers. Workflow can't be 60 engineers, so it's really 12 teams, given teams of 5. One takes the notifications engine and builds emails and slack notifications. They have an API for other teams, but this feature requir…

You just hand-waved their entire business as generic "workflow", picked up a few common technical issues (notifications, integration, support), and built teams around these issues. That indeed is going to hurt :-)

In team topologies, you'd consider them "complicated subsystems" teams. (See https://teamtopologies.com/key-concepts ) They may even be consider stream-aligned teams but they a feature crosses multiple streams to be valuable.

I used a workflow as a common type of SAAS business. My past and current companies could be considered such companies and had significant resources into such streams. Trying to enable the workflow of thousands of companies is a remarkably difficult problem.

But to take an example, look at monday.com as a workflow SAAS business that has a 9 billion dollar market cap. I would be surprised if they didn't have at least one team dedicated to tools for their support team.

Re: It's not microservice or monolith; it's cognitive load

#170
post #115

Earlier quoted context omitted.

A lot of factors come together whenever the term Microservie or Monolith arises: - Monolith has implicitly replaced the word "Legacy" system - Microservices were (/are still) the cool thing to do in a conference driven industry. They were championed by big tech because they desire granular and flexible deployment systems (whereas in smaller products your code just needs to run and everything else is just overhead). H…

> Monolith has implicitly replaced the word "Legacy" system Indeed, one couldn't possibly recommend a Monolith in 2023/4. A macroservice on the other hand ... Right. I'm off to polish my conference talk notes

That‘s why I think „monolith“ should only apply to the deployment strategy.

Building an application with „monolithic deployment“ is way closer to the truth than „building a monolith“.

Post reply on HN