Yes, because any artificial linkage that prevents one service from scaling independently from another violates the separation of concerns principal.
Is a shared database in microservices actually an anti-pattern?
61–70 of 93 posts
Re: Is a shared database in microservices actually an anti-pattern?
#62Earlier quoted context omitted.
I'm just an ignorant nobody, but why would you split a microservice off along a facet that would require transaction and referential integraty? Wouldn't you want to pick off pieces that could be truly independent and not need to care about transactions or referential integrity? otherwise, what problem are you actually solving by splitting the piece off?
If you never need to coordinate two or more microservices, then sure. But that means you're dealing with a monolith. In the original article, you could combine Orders and Users into a single microservice. That would resolve all the issues that are raised ... except that you might want reference the Users in a different context. At that point, you either have to split Users into their own microservice, or duplicate th…
> Part of the promise of microservice is that they're small modular independent components that you can connect and combine into higher-level services.
Sounds to me that the marketing copy for microservices is missing a crucial observation: the reason independent components aren't hard to work with in regular software is partly because everything runs single-threaded, or if you end up multithreading, the response is predictable and near real-time, the environment is reliable and under your control. These conditions essentially mask transactional and integrity issues, which only become apparent as you scale to multiple machines connected over a network.
Re: Is a shared database in microservices actually an anti-pattern?
#63Earlier quoted context omitted.
For Insert/update/delete, that makes sense. However, many times it makes sense to just do the join in the database rather than make an n^2 join in user space. This becomes problematic in collection cases - 100 cases belonging to 70 users. It's a quick left hand join but a slowdown in application space.
I don't disagree but I think in practice you're probably dealing with a single user at a time when working with orders. A query that referenced n users plus their orders is probably a reporting type query that could be supported in a data warehouse where all the tables could be joined together.
> query that referenced n users plus their orders is probably a reporting type query
No, there are enough other valid cases, e.g. display last customers and their orders or some simple "others have also bought". In practice you will need to join a lot spontaniously and can not just defer that to a data warehouse.
Re: Is a shared database in microservices actually an anti-pattern?
#64Earlier quoted context omitted.
I'm just an ignorant nobody, but why would you split a microservice off along a facet that would require transaction and referential integraty? Wouldn't you want to pick off pieces that could be truly independent and not need to care about transactions or referential integrity? otherwise, what problem are you actually solving by splitting the piece off?
If you never need to coordinate two or more microservices, then sure. But that means you're dealing with a monolith. In the original article, you could combine Orders and Users into a single microservice. That would resolve all the issues that are raised ... except that you might want reference the Users in a different context. At that point, you either have to split Users into their own microservice, or duplicate th…
Well, that isn’t a problem when you don’t care about the situation right now. For instance, let’s say you have VideoDescription, VideoSubtitling, and VideoContent as different microservices serving a description, the subtitles, and a handle to a list of content chunks. You need these things to line up (in that you may want all this for a single video). If VideoCentral says that you no longer have a movie it doesn’t matter. You can still serve video descriptions, subtitles, and content chunks until your view of the world changes. No big deal. If it is a big deal, then maybe the model doesn’t fit your problem. But for lots of software it really doesn’t matter. You don’t need instant consistency. If it’s consistent at some point that will do.
At no point will the writes insta-percolate. Instead you’ll push the writes into an event queue, they’ll execute eventually, and when the consuming services eventually update they’ll read the new result.
Re: Is a shared database in microservices actually an anti-pattern?
#65Earlier quoted context omitted.
I don't disagree but I think in practice you're probably dealing with a single user at a time when working with orders. A query that referenced n users plus their orders is probably a reporting type query that could be supported in a data warehouse where all the tables could be joined together.
> query that referenced n users plus their orders is probably a reporting type query No, there are enough other valid cases, e.g. display last customers and their orders or some simple "others have also bought". In practice you will need to join a lot spontaniously and can not just defer that to a data warehouse.
Most stores of large scale do not run online calculation or modeling for “others also bought”. It’s usually offline.
Re: Is a shared database in microservices actually an anti-pattern?
#66Earlier quoted context omitted.
The issue isn't about microservices, its about isolating dependencies between various components of the system. When any part of the system can reach into the user table then you've created a big ball of mud. Even with a monolith, you should have a defined contract between the user service and the rest of the system. That way you can start with a monolith and move to microservices later as needed. You can also change…
For Insert/update/delete, that makes sense. However, many times it makes sense to just do the join in the database rather than make an n^2 join in user space. This becomes problematic in collection cases - 100 cases belonging to 70 users. It's a quick left hand join but a slowdown in application space.
In Postgres you can use a foreign data wrapper to do such a join with a wide number of 3rd party data sources.
Re: Is a shared database in microservices actually an anti-pattern?
#67One pattern is to subscribe to an event bus and retain a read-only replica of your dependency's data. So the CustomerService publishes a CustomerModify operation which is picked up by the OrderService which then knows the new value of the customer's shipping address. If CustomerService wants to make backwards incompatible changes to its schema it should be handled by versioning so the CustomerService has to broadcast both versions of its message until such a time as all subscribers are using the updated version and then it can 'deprecate' the old message version. Ideally OrderService is using a library provided by CustomerService to handle applying CustomerModify messages so the next time OrderService is deployed (which should be continuously right?) it automatically picks up the new schema so this is a painless and automatic affair.
Re: Is a shared database in microservices actually an anti-pattern?
#68Yes, shared database is an anti-pattern in micro services architecture. If one shared database can serve your system well then you don't need microservices. You should build a monolith instead. The main reason for using microservices is scale. First ability to scale development teams and secondly ability to scale the infrastructure. With microservices you get vertical sharding out of the box. Yes it means dealing wit…
"If one shared database can serve your system well then you don't need microservices".
Exactly, nobody is even thinking about it anymore because "Twitter has microservices, Netflix uses MSA".
Microservice architecture comes with a cost, like more complex deployment strategy (there are more dependencies) and overhead in transaction management, not to mention debugging a problem which hits multiple domains/services.
Re: Is a shared database in microservices actually an anti-pattern?
#69There's a bit of an emperor's clothes problem with microservices. If microservices are never combined together, they are essentially monoliths under a cooler name. But as soon as you combine them, you run into the same problems that are solved by traditional shared database systems. Here are two: cross-microservice referential integrity, and cross-microservice transaction coordination, but there are plenty more. Take…
If you back your microservice architecture with a shared RDBMS and expect transactional and referential integrity, you've only scaled some parts of your system. Instead you should think about if A: you really need that scale ability, and B: the real implications of it.
So, in this customers / orders example what if you delete a customer? Ideally you don't, you keep the customer as long as you need the orders. You could perhaps anonymize it. And you define what happens if you cannot find the customer. Perhaps the orders should just be deleted? You could have a service running nightly that prune orders from customers that does not exist anymore. A cleanup service that checks for external dependencies and prune them as needed (note, this can be rather dangerous if done wrong).
For people that have formal education in distributed systems, or work with them, this seems very familiar, because it's some of the same things that make distributed systems hard. And that is what a microservice architecture is - a distributed system, which is why it scales well when done right.
This is also a problem we've mostly solved with CQRS and event sourcing, but it requires a lot of manual work and orchestration. It's hard and I can only say - you probably do not need a microservice architecture, and if you do, hire people with real experience in distributed system architecture. They're expensive, and if you can't afford it you do not need microservices.
Re: Is a shared database in microservices actually an anti-pattern?
#70The real answer: It depends.
But since most of the devs and architects fail to identify when you are allowed to do it, the rule of thumb is to not do it as it is a safer option.