Microservices are hard

If that’s indeed the most useful general thing that can be said about this question, then it means the microservices crowd has won, at least insofar as it deserved to win. The point is not that everything needs to be a micro-est possible service, the point is that we all now have applications built out of loosely-coupled services as a primitive in our reasoning toolbox.

(In a world where microservices haven’t yet won, we think RPC is obviously always a good idea and build our web apps as ISAPI modules and AOLserver extensions.)

I think very similar things when I hear Casey Muratori’s opinions on OOP[1], which he titled “getting rid of the OOP mindset” but I’d summarize as “absolutely do make an object or two if that fits your problem domain, but remember that it doesn’t have to and use your judgment”: that’s what OOP winning feels like, from the inside, once you’ve internalized it. Once people can look at a thing and think “you know what, that looks like an object”, OOP has won, even though people won’t always be right and not literally all the things are best modelled as objects.

(In a world where OOP—or lexical scope—hasn’t yet won, we read papers about Actors, struggle to understand what the authors could possibly mean, and implement a small language in order to have a version of those ideas we can play with. The language ends up being called Scheme.)

The same probably applies to structured programming, though it seems that structured programming deserved to win quite a bit more than the others, and though I don’t have easy access to pre-structured programming lore the same way I do to pre-OOP or pre-microservices lore.

In hindsight, old, good, now-accepted reasoning frameworks always seem to consist entirely out of trivially right stuff on one hand and outrageously wrong stuff on the other. That’s entirely by design and doesn’t mean the ancients were stupid—this is culture working correctly and well. It means that you, who has before only ever been in contact with the consensus that emerged out of their bitter struggles, have such a deep and implicit knowledge of that consensus that the good points that were once revolutionary sound obvious to you. You’re only seeing the bad points because those are the ones that didn’t find their way into the culture.

Scott Alexander’s metaphor of “philosophy in the water supply”[2,3,4] is the best explanation of this that I know.

[1] https://youtu.be/GKYCA3UsmrU

[2] https://slatestarcodex.com/2019/01/08/book-review-the-struct...

[3] https://slatestarcodex.com/2013/04/11/read-history-of-philos...

[4] https://slatestarcodex.com/2015/07/16/cbt-in-the-water-suppl...

I tend to split things up based on the resources required.

We had a decent sized data pipeline that was entirely microservice and serverless and it was a real joy to work with. - Our NLP code lived in a service that ran on GPUs - our ingest service used a high ram, but relatively simple CPU service to do in-memory joins cheaply and efficiently - we had a bunch of specialized query services that were just direct requests to AWS services, or a light lambda wrapper around a call to a service.

Coordinated things with airflow and it was very easy to maintain and scaling was pretty efficient since we just scaled the pieces that needed it without wasting money on unneeded compute.

I agree that I took it "a bit" far to make a point… that just ruined my case.

I do think that in a monolith you try to do most things yourself.

Take Auth for example. It's huge. And I think we more and more are excepting that even for a monolith we do not want to do that ourself.

If you start from scratch and your spec say that they want OIDC and SAML then most ppl will look for a saas service to get help.

Same with storing files, we would make use of S3 or similar.

But if we go to pre 2010. I think most people would try to both of these themself.

Im not saying that this is microservice at all, but I'm saying we are moving stuff that are commodity's out from the monolith compare to what we did 10-15 years ago. Because there are services that does a much better job for us.

And that was part of my point with the original comment. That trying to keep everything in one place is probably not the right choices.

Sorry for not being more precise on my previous reply.

At this point I think microservices architectures don't exist. As if a "micro"service is a goal on its own. I think any sane organization strives for a healthy trade-off between manageable code and separation of concerns.

I think the theory is when you want to change the dep you can do it in a config file instead of in the code. But I totally agree, I don't think the value is worth the cost.