I've settled on a compromise in this debate. Halfway between monoliths and microservices is the shared-library model. Instead of creating a microservice for your image processing, break it out into a standalone NPM or Composer or whatever module, then use that in your monolith. Gives you good separation of code and responsibilities, gives you good upgrade paths for your monoliths, avoids the overhead of microservices.
Monoliths Are the Future
71–80 of 567 posts
Re: Monoliths Are the Future
#72Earlier quoted context omitted.
Microservices are primarily about silo'ing different engineering teams from eachother. If you have a singular reporting database that a singular engineering team manages I'm not sure its a big deal. Reporting might be a "monolith" but the system as a whole isn't. Teams can still deploy their services and change their database schemas without stepping on eachother's toes.
> Teams can still deploy their services and change their database schemas No, because as soon as you change your schema, you have to plan ahead with the reporting team for starters. The reports still have to be able to work the same way, which means the data needs to be in the same format, or else the reports need to be rewritten to handle the changes in tables/structures.
Isn't that a weakness of any data dependency? I could push a new service that deprecates a field and supplies a replacement. I still have to communicate it and get downstream consumers to migrate. Or is the problem that reporting teams are looking at the low-level implementation details, rather than some stable higher-level interface? (I don't know how to avoid that when you're just pulling someone else's database directly or indirectly.)
Re: Monoliths Are the Future
#73As someone in the middle of destroying a monolith I hope the title is not true. Of course distributed-yet-still-deeply-coupled systems are possible they are at least harder to create. For me though the most important thing is grokability. Our monolith is to a point literally no one on earth can understand the whole thing. Even if the system is complex, the individual deployables being fully understood by some number…
> For me though the most important thing is grokability. Our monolith is to a point literally no one on earth can understand the whole thing.
Well, then you're really going to love it when the concerns are spread across different codebases connected by APIs!
Re: Monoliths Are the Future
#74I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…
I don't think you're right back to a monolith with centralized reporting. Remember, microservices doesn't mean JSON-RPC over HTTP. Passing updates extracted via change data capture and forwarding them to another reporting system is a perfectly viable interface. Data duplication is also an acceptable consequence in this design.
> Passing updates extracted via change data capture and forwarding them to another reporting system is a perfectly viable interface.
Right, that's the data warehouse method I described, keeping a central database in a reporting system. But now you just have to keep that database schema stable, because folks are going to write reports against it. It's a monolith for reporting, and its schema is going to be affected by changes in upstream systems. It's not like source systems can just totally refactor tables without considering how the data downstream is going to be affected. When Ms. CEO's report breaks, bad things happen.
Re: Monoliths Are the Future
#75I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…
No one has solved that problem and it sucks, and what ends up happening is you end up again porting that data from those disparate SQL and NoSQL databases either to a warehouse which is RDBMS or you put it into a datalake. That's again possible if you somehow manage to find all the drivers. You're doubly screwed if you have a hybrid - cloud and on-prem setup.
>> No one has solved that problem (...) you end up again porting that data from those disparate SQL and NoSQL databases either to a warehouse
That's exactly how that problem has been solved successfully for the past 20 years.
Re: Monoliths Are the Future
#76The answer is like often in between.
You might want to always breaker up a monolith but there is indeed little reason to do it with micro services. You could just use modules. Or better breaker it into a number of libraries with well defined interfaces, which you then compose into one monolith binary.
But there are very good reasons to split out some code into services (which might or might not be micro services, just not in the same process).
One is that it (easier) allows you to use more than one programming language. Normally you should avoid that, but there are sometimes reasons for it for example if 80% if what you need is implemented in a library available in that language.
Another one is you can have different reliability constraints for different parts of the system. (Like number of instances handling load parallel).
Another one is reuse between different systems (e.g. sharing of user management by e.g. using OpenId Connect).
Another one is that you can upgrade part of the system without stopping other parts.
....(a bunch more)
So in the end I would brake it in parts and compose that parts into a number of services but I would not bother with the whole "micro" part and other cloud marketing bs (because that's what it degraded to).
Re: Monoliths Are the Future
#77Earlier quoted context omitted.
Microservices are primarily about silo'ing different engineering teams from eachother. If you have a singular reporting database that a singular engineering team manages I'm not sure its a big deal. Reporting might be a "monolith" but the system as a whole isn't. Teams can still deploy their services and change their database schemas without stepping on eachother's toes.
> Teams can still deploy their services and change their database schemas No, because as soon as you change your schema, you have to plan ahead with the reporting team for starters. The reports still have to be able to work the same way, which means the data needs to be in the same format, or else the reports need to be rewritten to handle the changes in tables/structures.
That's no different than changing your API specification or refactoring your code or anything else. Ideally the entrypoints into your team's services should be considered hard contracts to the outside world and the rest of the organization. Changing the contract out from under your customer is not something that should ever be easy.
IMO the devops folks should define some standard containers that include facilities for reporting on low-level metrics. Most of the monitoring above that should be managed by the microservice owner. The messages that are consumed for BI and external reporting should not have breaking changes any more than the APIs you provide your clients should.
Re: Monoliths Are the Future
#78The author does not seem to understand when to correctly apply microservices. There are two basic use cases: 1) Different parts of your solution have different load patterns and it is economically beneficial to scale them at different rates and 2) Different teams need to be able to work & ship autonomously. It's not at all about technical merits or architectural beauty. It's about people and costs.
I kind of think the items (1) and (2) you list don't automatically mean micro-services, so much as they mean separation of concerns can be beneficial.
Isn't there room for a middle ground with modularity that can live in between a full blown monolith or a full blown microservices pattern, particularly for operations that are more medium scale?
Re: Monoliths Are the Future
#79I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…
Whether or not your system intentionally ended up with this architecture, GraphQL provides a unified way of fetching the data. You'll still have to implement the details of how to fetch from the various services, but it gives people who just want read access to everything a clean abstraction for doing so.
Re: Monoliths Are the Future
#80I have never worked on a real 'production' monolith. I wonder what i'm missing out on :)
For me a single monolithic stack is far more debugable.