Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monoliths Are the Future

changelog.com

Post reply on HN
Print view View original

181–190 of 567 posts

Re: Monoliths Are the Future

#181

Post by tynpeddler »

There are two big reasons to go to microservices (note that the exact definition of microservice can vary a lot).

1. Organizational streamlining. If the team working on the monolith becomes to large, then coordinating and pushing out changes quickly can become incredibly difficult. One rule of thumb I've heard is the two pizzas rule. If two pizzas can't feed the team working on a system, it's time to break up the system.

2. Horizontal scaling. If some components of your workflow require much more computing power than others, then it makes sense to break up your system to move computationally intensive tasks to their own services.

While there are lots of other decent reasons to break up a system, if you can't invoke at least one of the two above reasons, you may be shooting yourself in the foot. I think he's dead on when he points out that if you don't have engineering discipline in the monolith, then you won't have it in the microservices.

Re: Monoliths Are the Future

#182

Post by BrentOzar »

jsjohnst wrote: post #129
BrentOzar wrote: post #107

Earlier quoted context omitted.

> But I’d argue Monoliths don’t have anything inherent to them which makes reporting easier. It's easier to join tables in databases that live on a single server, in a single database platform, than it is to connect to lots of different data sources that live in different servers, possibly even in different locations (like cloud vs on-premises, and hybrids.)

> It's easier to join tables in databases that live on a single server, in a single database platform 1) what if your data set is larger than you can practically handle in a single DB instance? 2) nothing about a monolith implies you have a single data platform, let alone a single DB instance

> 1) what if your data set is larger than you can practically handle in a single DB instance?

I have clients at 10TB-100TB in a single SQL Server. It ain't a fun place to be, but it's doable.

Re: Monoliths Are the Future

#183

Post by BrentOzar »

jsjohnst wrote: post #138
mwigdahl wrote: post #108

Earlier quoted context omitted.

I don't think Brent was necessarily saying it was a good idea, just something that is commonly seen. At my own company we try hard to get customers to run reports against replicas/extracts rather than production, but some customers insist that they absolutely _need_ to pull in live data from production. So they run complex reports against a production OLTP schema and wonder why the system is running slow...

> I don't think Brent was necessarily saying it was a good idea Reading other comments Brent’s made here, I’m not so sure.

> Reading other comments Brent’s made here, I’m not so sure.

No no, it's not a good idea, but it's just the reality that I usually have to deal with. I wish I could lay down rules like "no analyst ever gets the rights to query the database directly," but all it takes is one analyst to be buddy-buddy with the company owner, and produce a few reports that have really high business value, and then next thing you know, that analyst has sysadmin rights to every database, and anybody who tries to be a barrier to that analyst's success is "slowing the business down."

Re: Monoliths Are the Future

#184

Post by mewpmewp2 »

echelon wrote: post #165
adamc wrote: post #146

Earlier quoted context omitted.

I do not claim expertise here, but it would seem like microservices would add significant performance costs. Stitching together a bunch of results from different microservices is going to be a LOT more expensive than running a query with joins.

Humans are the most expensive part of the system. You have to make it easy for humans to understand and change the system, and at the end of the day that's the number one thing to optimize for. This is why microservices are compelling. But to speak directly to your concern, you have to think about service boundaries and granularity correctly. Nobody is saying make a microservice out of every conceivable table. Think…

In my experience it is a lot more difficult to navigate around all the different microservices to understand what needs to be done compared to being in a monolith where you can jump from file to file.

Also then what also happens is microservices are created using different languages which in turn adds so much complexity to understand what is going on on the whole big picture level.

And code gets repeated a lot more. If there is change in a microservices or update everyone will need to figure out what services depend on and how they will have to adapt. With monolith you can just use your IDE to see what will break if you make a change. So much repeated business logic. Creating a new feature involves having to have many meetings to figure out what services in which way have to be updated.

It is crazy mess in my opinion.

I have been with a company that had monolith application which they split up to more than 15 services (some python, some js, Scala, Java, etc...). Monolith still is used for some parts that are not migrated. I was working on single service having no idea how the whole system worked together. Then I had to do something in the old parts and I very quickly got an understanding how everything works together.

Re: Monoliths Are the Future

#185

Post by allover »

BrentOzar wrote: post #179
MadWombat wrote: post #131

Earlier quoted context omitted.

> Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting? One of the key concepts in microservice architecture is data sovereignity. It doesn't matter how/where the data is stored. The only thing that cares about the details of the data storage is the service itself. If you need some data the service operates on for reporting purposes, make an API that gets you this data and…

> But you do not simply go and poke your reporting fingers into individual service databases. In a good microservice architecture you should not even be able to do that. I agree. In a monolith architecture, though, you CAN do that (and many shops do.) That's where their pains come from when they migrate from monolith to microservice: development is easier, but reports are way, way harder.

> when they migrate from monolith to microservice: development is easier [...]

Not even that -- that idea is still highly debatable.

I would argue that it absolutely isn't easier, and the stepping-back-in-time of developer experience is one of the biggest problems with microservices.

Microservices in general, are way, way harder.

Re: Monoliths Are the Future

#186

Post by Yeroc »

gunnarmorling wrote: post #170
rjkennedy98 wrote: post #19

Earlier quoted context omitted.

This is what Kafka is for. You put Kafka on top of your database to expose data and events. Now BI can take the events put them into their system as they want.

Shameless plug: feeding data from operational databases to DWH is one main use case of change data capture as e.g. implemented by Debezium ( https://debezium.io/ ) for MySQL, Postgres, MongoDB, and more, using Apache Kafka as the messaging layer. Disclaimer: working on Debezium

Curious how support is for commercial databases like SQL Server and Oracle? In the larger, older, enterprises these databases are rampant...

Re: Monoliths Are the Future

#187

Post by zelly »

one2know wrote: post #83

Perhaps the solution is a single server where one can drop a zip file containing their microservice and have it automatically deploy without the hassle of setting up a new pod/server/container. Congratulations you have reinvented java servlets 1.0 circa 1996.

My prediction: I think soon we will see public clouds provide WebAssembly hosting, where you upload a single WebAssembly binary and it just works. Your whole application is compiled and bundled to that WebAssembly. There will even be glue code that helps migration from a microservices architecture by stitching them together in parallel containers, all compiled down to WebAssembly of course.

Re: Monoliths Are the Future

#188

Post by ForHackernews »

WoahNoun wrote: post #126
BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

I'm going to disagree heavily here. The world of cloud computing, microservices, and hosted/managed services has made the analyst and data engineers job easier than ever. If the software team builds a new dynamodb table, they simple give the AWS account for the analytics team the appropriate IAM permissions and the analytics team will set-up an off-peak bulk extract. A single analyst can easily run an entire data war…

What/where do you run this mythical one-analyst pipeline, though? Is that in cloud services too? Airflow? Kubeflow? Apache Beam? It sounds like you're just pushing the problem around.

Re: Monoliths Are the Future

#189

Post by sequoia »

MadWombat wrote: post #131
BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

> Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting? One of the key concepts in microservice architecture is data sovereignity. It doesn't matter how/where the data is stored. The only thing that cares about the details of the data storage is the service itself. If you need some data the service operates on for reporting purposes, make an API that gets you this data and…

> you do not simply go and poke your reporting fingers into individual service databases

Side point: This is a needlessly hostile and unprofessional way to refer to a colleague. Remember that you and the reporting/analytics people at your company are working towards the same goals (the company's business goals). You are collaborators, not combatants.

You can express your same point by saying something like "The habit of directly accessing database resources and building out reporting code on this is likely to lead to some very serious problems when the schemas change. This is tantamount to relying upon a private API." etc.

We can all achieve much more when we endeavor to treat one another with respect and assume good intentions.

Re: Monoliths Are the Future

#190

Post by willvarfar »

BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

https://prestosql.io/

It can access all those different databases.

You can also make your own connectors that make your services appear as tables, which you can query with SQL in the normal way.

So if the new accounts micro-service doesn't have a database, or the team won't let your analysts access the database behind it, you can always go in through the front-door e.g. the rest/graphql/grpc/thrift/buzzword api it exposes, and treat it as just another table!

Presto is great even for monoliths ;) Rah rah presto.

Post reply on HN