Live data from Hacker News

It's not microservice or monolith; it's cognitive load

fernandovillalba.substack.com

181–190 of 195 posts

Re: It's not microservice or monolith; it's cognitive load

#181

Earlier quoted context omitted.

With no or minimal communication how is it possible to build anything new? In real world if you need to build something new and it requires 10 different companies to do that, it would be virtually impossible.

The same way the economy has always built something new. We have thousands of years of experience in doing that. You don’t try to build something new with 10,000 developers, if that’s where you have become confused.

I think the confusion is because "something new" isn't necessarily a completely new, disconnected product. It could be something like adding multi-lingual support to an existing product. In that case, you often do need to coordinate across many teams to get the new feature implemented, which can be a huge pain in a micro-service architecture.

Re: It's not microservice or monolith; it's cognitive load

#182
post #172

Earlier quoted context omitted.

I don't read cockiness or narcissism in his posts. Having just started reading Clean Architecture by Robert C. Martin two days ago (and hoping to complete it tonight, then planning on re-reading it again), it seems to me that hmeh has learned and PRACTICED what is discussed in that book. I wish I were in their shoes!

Let's just say that that sphere of the software dev community isn't exactly universally seen as competent work, more like the orthodoxy of ~15 years ago that's now increasingly, and in my opinion rightfully, being put into question for its bad complexity and performance trade-offs when applied beyond textbooks and conference talks. And this is being diplomatic compared to others. I'm certainly not saying there isn't…

> The option to do non-clever code is seemingly insulting to the self-professed architect, but it is in my own experience at least, vastly more maintainable and productive, not because it results in "clean" code - it's usually pretty dirty and nothing to write home about, but because reading even a page-long flat procedural function that's just doing its thing, is much more straight-forward than ping-ponging between some clever architect's conglomeration of decaying design patterns.

With all due respect, if you think you are talking about anything that I was attempting to describe, you are not. Everything that we do is based on simplicity and building the most simple things that we can. We may have different definitions of simplicity however. For mine, check out Rich Hickey's "Simple Made Easy" talk if you haven't seen it.

We simplify things by de-complecting or, avoiding entanglement. Having separate repositories is just one way to re-enforce the simplification that we put in place with design. We call that structural design (because it's the design of the actual structure of the application).

> on that note, it sounds like hmeh is trying to work around this problem with abstractions by doing a significant amount artificial partitioning.

I can guarantee you that none of the partitioning is "artificial". I think at this point that you have projected so much of your experience into what I am saying that it's impossible for you to see anything I say in any other light. This, by the way, is what I mean by tacit knowledge. I say X, and you either hear Q, or you hear XIENRSTIENRSTIERNST, either way, you cannot possible understand what I am saying. This isn't a negative reflection on you, or a positive reflection on me. It's simply the way that tacit knowledge works, and it's why most "internet arguments" are wastes of time.

> People promoting these things will of course then just blame it on the dev and call them average, or lacking in experience, because they can't identify and combine the various clever patterns correctly, they "are doing it wrong", and that's where I believe the above narcissistic tendencies I called out comes from.

And yet, it is literally lack of experience (and ability) that prevents people from doing things they cannot yet do. That's how expertise works. Pointing that out is not narcissism. A gardener does not have the experience (or training) necessary to do brain surgery. This isn't a mark against the gardener. The difference in our industry is that it is filled with expert beginners. People who think that because they make 6-figure salaries after their limited education and experience they can now understand everything there is to understand about software design. This just isn't the case. If a person sees that as a slight to a fellow human, that says more about them than it does about the person pointing to that reality.

Re: It's not microservice or monolith; it's cognitive load

#183
post #161

Earlier quoted context omitted.

> Let's see here: Yes, let's. I have no problem with push back. > You have experienced what happens when web developers cosplay as software architects. I agree that this is overly snippy to the point of being counter-productive. It represents a particular emotional frustration with the state of our industry. We knew what ended up being microservices (web API servers making web API calls to other web API servers) was…

Answered somewhat what I mean by at least the "code smells" and idiosyncrasies (competency was probably too harsh with given the amount of info) in a reply to another user. > Your arrogance precludes a fruitful discussion. This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent. Not sure what can come…

> This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent.

I touched on this in my reply to the other user, but at this point, I believe you may just not know what tacit knowledge is. You continue to think that I am pointing to our own brilliance. I'm not. I'm calling a spade a spade. I recognize what it takes to gain particular types of knowledge (tacit, or subtle knowledge), and I recognize that it's this reality that prevents most conversations about techniques from being fruitful.

Each participant will put their own experience behind the meaning of their words (and worse, their conversation partner's words) and it will prevent them from recognizing what one another are saying. The only way to have a fruitful discussion is for both sides to be capable of recognizing when that is happening. Since, in my experience, most people aren't -- they'd rather die on their hill than recognize that the person they are talking to is simply on the same hill but sees it differently, or they are on a different hill that really is better, but it cannot be seen as such yet, because it is over the horizon of their knowledge.

I don't actually know if you are interested in understanding this more or if you joined the conversation just to try to put me in my place, but if you are, here are some articles that may help:

https://madabout.software/articles/subtle-knowledge-crude-kn...

https://madabout.software/articles/design-is-subtle-knowledg...

Once you can recognize that there is subtle knowledge you might actually see my pointing to it as attempting to keep the conversation from devolving into exactly the type of thing that it tends to devolve into. Or not.

Re: It's not microservice or monolith; it's cognitive load

#184
post #52

Earlier quoted context omitted.

I don't disagree completely with the comment, but I would possibly change it from "comply with Conway" tô "embrace Conway".

If you don't disagree, then I assume you have a strong understanding of structural design. I assume that you recognize that Conway's law is more of a curse, and a warning than it is something to embrace. I assume that you recognize that the only possible way to "embrace conway's law" and simultaneously recognize structural design would be to constantly be firing or otherwise disbanding entire teams as components get…

No, not at all. I see why you think of it as a curse, I don't enjoy it as a resulta, but it's an observation of a pervasive pattern, not a design decision. It emerges from the way humans communicate and organize themselves.

You can either fight it agressivelly and try to minimize it, or you can "embrqce" it and try to adapt. I could force it into a disability analogy:

if you become suddenly deaf, you can agressivelly try to reverse it by all medical means possible, and you might succeed. Or you might not, then it would be better for you to accept the fact you are deaf and design your life around this disability as a means to adapt yourself to it.

That's how I see microsservices, too: accepting that large enough organizations cannot communicate as effectivelly and efficiently as small ones, therefore designing systems to minimize the impacts of these communication failures.

Re: It's not microservice or monolith; it's cognitive load

#185

Earlier quoted context omitted.

They are a constant sources of bugs, but it's not really an architecture matter.

It is because that's literally an architectural choice which prevents you from easily moving out a module from your "monolith" to another machine on the network, and causes the bugs. The language & memory architecture... are an architecture matter.

Yes, but not necessarily: with rust you can put stuff on the heap yet it's still ownership-checked just like the stack is scoped in other languages. Conceptually I mean.

Re: It's not microservice or monolith; it's cognitive load

#186
post #183

Earlier quoted context omitted.

Answered somewhat what I mean by at least the "code smells" and idiosyncrasies (competency was probably too harsh with given the amount of info) in a reply to another user. > Your arrogance precludes a fruitful discussion. This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent. Not sure what can come…

> This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent. I touched on this in my reply to the other user, but at this point, I believe you may just not know what tacit knowledge is. You continue to think that I am pointing to our own brilliance. I'm not. I'm calling a spade a spade. I recognize what…

> I believe you may just not know what tacit knowledge is

Not sure where you got that from, haven't said anything about it nor did I include that in my quotes.

I'm well aware of it. But if I have one comment on it would be that I see it more as something an experienced person (or someone with a natural knack for it) makes use of under-the-hood, the resulting quality of the output however can usually be recognized by everyone, not something reserved for the "blessed ones". Take the redis source code for example (quite a few years since the last time I read it though). The author clearly has this skill, but one doesn't have to possess that to recognize the code quality (and btw, iirc without any mentions of a pile of design patterns/methodologies, just "doing it", but to each their own).

So therefore I'm a bit suspicious of anyone that claims that their idiosyncratic setup is actually simple if you just have experience enough to be able to judge it. I'm not saying that everything can be obvious at first glance but the vagueness triggers my radar after being worn down by experiencing way too much over-engineering motivated by self-serving vagueness and/or word salads ("baffle them with bullshit").

That said, if a code base keeps requiring you to make the correct design decision using subtle knowledge, that's a fragile situation and something seems off. You've probably already made the code base too complicated, my prejudices (somewhat confirmed by the language used in one the articles you linked [1]) would be through a pile of design patterns inspired abstractions that now everyone needs to be able to juggle at all times.

[1] "For example, the Tell, Don’t Ask principle can’t be expressed directly in terms of Push, Don’t Pull, which has a more common name: encapsulation. And each one of these qualities reflects cohesion and coupling. And furthermore, cohesion and coupling are inter-related and affect each other. Afference and efference are kinds of coupling. Afference is inbound coupling, and efference is outbound coupling"

Re: It's not microservice or monolith; it's cognitive load

#187
post #183

Earlier quoted context omitted.

> This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience as the answer to why your setup is not suspicious, but excellent. I touched on this in my reply to the other user, but at this point, I believe you may just not know what tacit knowledge is. You continue to think that I am pointing to our own brilliance. I'm not. I'm calling a spade a spade. I recognize what…

> I believe you may just not know what tacit knowledge is Not sure where you got that from, haven't said anything about it nor did I include that in my quotes. I'm well aware of it. But if I have one comment on it would be that I see it more as something an experienced person (or someone with a natural knack for it) makes use of under-the-hood, the resulting quality of the output however can usually be recognized by…

> Not sure where you got that from, haven't said anything about it nor did I include that in my quotes.

I got it from here:

> This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience ...

and

> Being so cocksure and backing it up with statements that you need to be as brilliant as you to "understand" is such a laughingly large red flag...

Given that at no point did I point to our "brilliance", I assumed you refererring to my pointing to tacit knowledge:

> It's honestly a hard question to answer because the real answer requires tacit knowledge...

> ... because much of what we do requires tacit knowledge to see the benefit of ...

So if I'm mistaken, I apologize, but please do point out where I claimed brilliance. I am certainly claiming experience, and expertise, but those are earned, as they are in any profession. It's also true that no one in this conversation (including you) other than me have actually seen our code, and therefore would be ill-equipped to judge it. That is, they do not have the experience of our code base. Rushes to judge it based on pre-conceived notions only reflect a lack of dilligence, integrity, and/or awareness.

> the resulting quality of the output however can usually be recognized by everyone...

Have you seen our code? Or have you seen a few mentions of some of the things that we do and you are using that to fabricate an idea of what our code is?

> So therefore I'm a bit suspicious of anyone that claims that their idiosyncratic setup is actually simple

You are of course free to be suspicious. A significant part of the experience necessary to judge it would be to actually see it. We are, at the end of the day, dancing about architecture. You cannot see our code, you cannot see our actual set up, which typically would mean that one would have essentially nothing to say about it. If you'd like to refute particular practices I mention, that's fine. But please, keep your presumptions to yourself, or at least ask them in the form of a question (e.g., "artificial partitioning").

> ...would be through a pile of design patterns inspired abstractions that now everyone needs to be able to juggle at all times.

We have relatively junior developers on our team. There are a small handful of common patterns that are used repeatedly throughout our code base. We strive to eliminate special (unnecessary) variatation so there are always exemplars and/or documented norms. We don't have 10 varieties of "service objects". We don't have callback hell. They do not struggle with these things. They struggle with other things, as they are relatively junior, but we support them. Again, frankly, you have absolutely no idea what you are talking about and you are continuing to attempt to make up for your ignorance (of our code base and what I am saying) with hubris and presumption.

I just saw this edit from you:

> That said, if a code base keeps requiring you to make the correct design decision using subtle knowledge, that's a fragile situation and something seems off.

You and I have two different ideas of expertise and software design and they are irreconciable. Maybe in 15 years you can look back on this conversation and recognize that there was always land beyond the horizon. Or maybe not. You are the doctor that is refusing to wash his hands before surgery because you still believe in bad humors. There are things you do not know, and you are stubbornly refusing to recognize them. You have probably worked on some pretty horrific code bases, and you can probably back up every one of the things you are projecting onto me with personal experience, but you have drawn the wrong ultimate conclusion. You have drawn the conclusion that software design does not exist and does not have consequences and that anyone who claims to do it is a charlatan. That may even be true much of the time (goodness knows that's what I'm saying about many people who claim to be doing software design). Judge me as harshly as you wish for making this assessment. Thankfully, we do not work together and we will be unlikely to cross paths again in the future.

Re: It's not microservice or monolith; it's cognitive load

#188
post #187

Earlier quoted context omitted.

> I believe you may just not know what tacit knowledge is Not sure where you got that from, haven't said anything about it nor did I include that in my quotes. I'm well aware of it. But if I have one comment on it would be that I see it more as something an experienced person (or someone with a natural knack for it) makes use of under-the-hood, the resulting quality of the output however can usually be recognized by…

> Not sure where you got that from, haven't said anything about it nor did I include that in my quotes. I got it from here: > This was specific to the tendencies in other replies to defer to your (or your team's) brilliance/experience ... and > Being so cocksure and backing it up with statements that you need to be as brilliant as you to "understand" is such a laughingly large red flag... Given that at no point did I…

> Given that at no point did I point to our "brilliance", I assumed you refererring to my pointing to tacit knowledge

I deduced it from your general tone. But you're probably right that the tacit was part of it.

> Have you seen our code?

No, but it was a general comment, hinted by providing the redis source code as an example.

> and presumption.

Yes, it was prejudicial, as stated. I hope I'm wrong.

> You and I have two different ideas of expertise and software design and they are irreconciable.

Agree.

> Maybe in 15 years you can look back on this conversation and recognize that there was always land beyond the horizon. Or maybe not. You are the doctor that is refusing to wash his hands before surgery because you still believe in bad humors. There are things you do not know, and you are stubbornly refusing to recognize them.

And here's why I said your arrogance would make it unfruitful. Because in the end you're of course objectively right, and I'm objectively wrong and will see my errors in due time. Sigh.

> Thankfully, we do not work together and we will be unlikely to cross paths again in the future.

The feeling is mutual!

Re: It's not microservice or monolith; it's cognitive load

#189
post #76

Reading the responses/comments in here, a question arises... Are things really that black and white as people here paint them - with monoliths unavoidably and necessarily becoming a tangled mess of spaghetti, and micro-services being the singular, only way to achieve clean separation, autonomy and partition? With microservices inevitably being an organizational nightmare that leads to problems in inter-team coordinat…

The worst WTF garbage software I have seen in my 30+ years career is an old micro-services system maintained by a team at work.

It uses more than 100 micro-services to implement a system that would have been so much simpler if implemented as a monolith. With hundreds of scripts to try and manage the complexity. With frequent timing bugs (a single micro-service delay causing a cascade of timed out micro-services bringing the system down) and almost impossible to fix complex error scenarios that will make experienced developer heads spin.

In contrast, the systems I work on are million+ lines C/C++ monoliths that are orders of magnitudes simpler to maintain. With orders of magnitude fewer bugs in production. All implemented as a collection of well designed libraries.

IMHO the moment you split a monolith into 100+ micro-services, the number of failure scenarios dramatically increase because of the added complexity of message exchanges, message failures, queuing delays, circular dependencies etc.

In other words, if you don’t even have the skills to implement a monolith successfully, then stay the hell away from micro-services.

Re: It's not microservice or monolith; it's cognitive load

#190

Earlier quoted context omitted.

The same way the economy has always built something new. We have thousands of years of experience in doing that. You don’t try to build something new with 10,000 developers, if that’s where you have become confused.

I think the confusion is because "something new" isn't necessarily a completely new, disconnected product. It could be something like adding multi-lingual support to an existing product. In that case, you often do need to coordinate across many teams to get the new feature implemented, which can be a huge pain in a micro-service architecture.

No more than it is a pain in a service architecture – being the exactly same thing, just in a different economy. As always, if a service doesn't do what you need, you build your own in-house. Just because there is a product on the market is that is kind of, sort of, but not really, what you need doesn't mean you must use it.
Post reply on HN