Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is a shared database in microservices actually an anti-pattern?

hackernoon.com

Post reply on HN
Print view View original

71–80 of 93 posts

Re: Is a shared database in microservices actually an anti-pattern?

#71

Post by FrankyHollywood »

thoman23 wrote: post #43

My understanding is that the point of microservices isn't to just move what would have been a relational join in SQL to an equivalent join in the service layer, which is what the author is implying. Instead, the order service should already have the subset of user data it needs to perform its function. If a user name changes, for example, that would trigger an event that gets consumed by a handler in the order domain…

you're right, I'm in the middle of a big project where the lead engineer decided 'data duplication is forbidden'. The mess it created is terrible, both in complexity and in performance.

I can't imagine microservice architecture will ever work in a complex domain without every service having most of its dependent data in its own store.

What would help if people don't look at it as 'duplication' but rather 'caching'.

Re: Is a shared database in microservices actually an anti-pattern?

#72

Post by barrystaes »

> Is a shared database in microservices actually an anti-pattern?

No, unless you dont want (or care about having) a single source of truth. If state is distributed its harder to backup/restore/rewind or even query atomically and reproducibly.

You could still make the single database sharded and duplicated though, but still: in most cases its still one shared database. Even storing Files outside of a DB is just sharding, important thing is the DB refers to the file and still is the single source of truth.

And when you want vertical segmentation see designs like multi-tenant, but its still not a database per microsevice: quite the opposite.

When you dont want a single source of truth you could do without it ofcourse.

Re: Is a shared database in microservices actually an anti-pattern?

#73

Post by LaserToy »

From my expirience, at some scale (system and organization) it became a easier to solve technical issues of having multiple DBs than dealing with bottlenecks on 1 shared DB. We had a fleet mucroservices that used 3 semishared DBs (1 for 1 domain, 1 for another domain and 1 shared). It worked ok for a year and now we run 40 DBs. It is harder to maintain and harder to develop against, but we are not constantly stuck in redeploying half of the stack because scheme just changed.

So, no silver bullet. You need to use something that makes sense, not what is fancy today.

Re: Is a shared database in microservices actually an anti-pattern?

#74

Post by tirumaraiselvan »

hvidgaard wrote: post #69
quizotic wrote: post #46

There's a bit of an emperor's clothes problem with microservices. If microservices are never combined together, they are essentially monoliths under a cooler name. But as soon as you combine them, you run into the same problems that are solved by traditional shared database systems. Here are two: cross-microservice referential integrity, and cross-microservice transaction coordination, but there are plenty more. Take…

If you back your microservice architecture with a shared RDBMS and expect transactional and referential integrity, you've only scaled some parts of your system. Instead you should think about if A: you really need that scale ability, and B: the real implications of it. So, in this customers / orders example what if you delete a customer? Ideally you don't, you keep the customer as long as you need the orders. You cou…

How does CQRS solve the problem of referential integrity?

Re: Is a shared database in microservices actually an anti-pattern?

#75

Post by hvidgaard »

tirumaraiselvan wrote: post #74
hvidgaard wrote: post #69

Earlier quoted context omitted.

If you back your microservice architecture with a shared RDBMS and expect transactional and referential integrity, you've only scaled some parts of your system. Instead you should think about if A: you really need that scale ability, and B: the real implications of it. So, in this customers / orders example what if you delete a customer? Ideally you don't, you keep the customer as long as you need the orders. You cou…

How does CQRS solve the problem of referential integrity?

With a distributed system, there is no such thing as referential integrity. You can only mitigate the issue, and CQRS is good at that.

Re: Is a shared database in microservices actually an anti-pattern?

#76

Post by tirumaraiselvan »

hvidgaard wrote: post #75
tirumaraiselvan wrote: post #74

Earlier quoted context omitted.

How does CQRS solve the problem of referential integrity?

With a distributed system, there is no such thing as referential integrity. You can only mitigate the issue, and CQRS is good at that.

Can you give me a concrete example of using CQRS in such a setting and the problem it is solving?

Re: Is a shared database in microservices actually an anti-pattern?

#77

Post by hvidgaard »

tirumaraiselvan wrote: post #76
hvidgaard wrote: post #75

Earlier quoted context omitted.

With a distributed system, there is no such thing as referential integrity. You can only mitigate the issue, and CQRS is good at that.

Can you give me a concrete example of using CQRS in such a setting and the problem it is solving?

You have to use event sourcing as well, CQRS alone does not solve it.

First you have to decide what trade offs you want. Ideally you will not expose any events from a service, but that is not realistic. And since the two services have some degree of connection, let's make the trade off that we want to expose events of creation and deletion of customers so other systems can keep track of a current list of customers. We utilize an at-least-once delivery mechanism of events.

The orders service would subscribe to the two events, and maintain an internal list of currently active customers. You cannot create orders for customers that does not exists, and when a customer is deleted, you can do what you need to do with the orders.

Re: Is a shared database in microservices actually an anti-pattern?

#78

Post by zaarn »

eweise wrote: post #53
ebiester wrote: post #42

Earlier quoted context omitted.

For Insert/update/delete, that makes sense. However, many times it makes sense to just do the join in the database rather than make an n^2 join in user space. This becomes problematic in collection cases - 100 cases belonging to 70 users. It's a quick left hand join but a slowdown in application space.

I don't disagree but I think in practice you're probably dealing with a single user at a time when working with orders. A query that referenced n users plus their orders is probably a reporting type query that could be supported in a data warehouse where all the tables could be joined together.

"Show content your friends liked" is an easy example where you want an in database join, in userspace this operation would be incredibly slow.

Re: Is a shared database in microservices actually an anti-pattern?

#79

Post by HelloNurse »

hvidgaard wrote: post #69
quizotic wrote: post #46

There's a bit of an emperor's clothes problem with microservices. If microservices are never combined together, they are essentially monoliths under a cooler name. But as soon as you combine them, you run into the same problems that are solved by traditional shared database systems. Here are two: cross-microservice referential integrity, and cross-microservice transaction coordination, but there are plenty more. Take…

If you back your microservice architecture with a shared RDBMS and expect transactional and referential integrity, you've only scaled some parts of your system. Instead you should think about if A: you really need that scale ability, and B: the real implications of it. So, in this customers / orders example what if you delete a customer? Ideally you don't, you keep the customer as long as you need the orders. You cou…

The system can simply mark the "deleted" customer as a former customer and add records of their dismissal without any referential integrity problems. "Deleting" an entity doesn't mean that it should immediately vanish without a trace from the database, leaving a wake of destruction. It's only a business-level change: we won't accept further orders from deleted customers, there might be something nasty to do to their outstanding orders, and so on.

Re: Is a shared database in microservices actually an anti-pattern?

#80

Post by HelloNurse »

gcufhg wrote: post #23
ChicagoDave wrote: post #9

Earlier quoted context omitted.

The only thing that needs a relational data store are reports. Operational data stores should only be concerned with their own domain and expose behaviors and events. There are exceptions, but if you’re building a complex system, monolithic architectures have a known lifetime while MSA’s tend to mitigate long term coderot. It’s hard to see this until you’ve built domain driven micro service based systems properly.

You could build a separate ReportService which replicates only the data that's needed for a report in RDBMS.

With a sane database design, the Report service can read user details of all users of interest from the user service and ask the Orders service for the orders of each user (a presumably efficient query), without replicating data.
Post reply on HN