Live data from Hacker News

Monoliths Are the Future

changelog.com

121–130 of 567 posts

Re: Monoliths Are the Future

#121
post #61

Earlier quoted context omitted.

Agree with your last point. And as someone who really liked old school callback/prototype/closure based JavaScript, I can say not only would these people have been better off using a better language, they also ruined JavaScript for the things it was great at.

You can still do all that in JS, novel syntactic sugar aside.

He/She cannot do that, at least at their employer, because people would throw up at the sight of the code and cry until its rewritten.

As an example slip some DOM methods into your code and watch people go into convulsions like an angry zombie on cocaine.

Re: Monoliths Are the Future

#122

With respect to the author, who probably is a much smarter person than I am, this is yet another in a long, long series of HN articles that should be grouped under "I don't know what the hell X is, but I was an expert in it, and I can tell you it sucks" I've seen X be a dozen things: UML, databases, User Stories, Functional Programming, Testing... It's too much to list. Yes. If you do it that way it will hurt, and yo…

"who probably is a much smarter person than I am" -- I have a different take-away from the quality of that writing. I'm also no fan of the kubernetes code.

> I have a different take-away from the quality of that writing.

It's an excerpt from a podcast.

Re: Monoliths Are the Future

#123
The argument I have always heard is that microservices are a way to solve problems arising in large organizations with multiple teams trying to push code all in to the same repository and managing the deployments from such an organization. I get why separating the different teams' services in to separate silos helps, but the fact that the solution puts a network between the different parts where there was none feels wrong. You are trading one problem for another problem. Networks are slower and less reliable than calling a function in the same application.

Microservices may be the solution to this problem right now, but I believe someone is going to come up with some other solution (tooling etc) that allows you to get the benefits of migrating to microservices without having to add a unnecessary network layer just to solve an organizational problem.

Re: Monoliths Are the Future

#124
post #71

I've settled on a compromise in this debate. Halfway between monoliths and microservices is the shared-library model. Instead of creating a microservice for your image processing, break it out into a standalone NPM or Composer or whatever module, then use that in your monolith. Gives you good separation of code and responsibilities, gives you good upgrade paths for your monoliths, avoids the overhead of microservices…

Just like we were doing with COM and DLLs more than twenty years ago! It worked very well.

Re: Monoliths Are the Future

#125

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

This is a solved problem, "Data Engineering" teams solve this by building a data pipeline. It's not for all orgs, but for a large org, this is worth doing right.

The remark wasn't that there weren't solutions, but that it is still a problem that needs a solution. Storing all data within a single database is a much simpler way to get started if you want to run analytic queries. You spin up a read slave that is tuned for expensive queries, rather than OLAP ones.

Re: Monoliths Are the Future

#126

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

I'm going to disagree heavily here. The world of cloud computing, microservices, and hosted/managed services has made the analyst and data engineers job easier than ever. If the software team builds a new dynamodb table, they simple give the AWS account for the analytics team the appropriate IAM permissions and the analytics team will set-up an off-peak bulk extract. A single analyst can easily run an entire data warehouse and analytics pipeline basically part time without a single server using hosted services and microservices. With a team of analysts, the load sharing should be such that the ETL infrastructure is only touched when adding new pipelines or a new feature transformation.

And for data scientists working on production models used within production software, most inference is packaged as containers in something like ECS or Fargate which are then scaled up and down automatically. Eg, they are basically running a microservice for the software teams to consume.

Real time reporting, in my opinion, is not the domain of analysts; it's the domain of the software team. For one, it's rarely useful outside of something like a NOC (or similar control room areas) and should be considered a software feature of that control room. If real-time has to be on the analysts (been there), then the software team should dual publish their transactions to kinesis firehouse and the analytics team can take it from there.

Of course, all of this relies heavily on buy-in to the world of cloud computing. Come on in, we all float down here.

Re: Monoliths Are the Future

#127

Earlier quoted context omitted.

No one has solved that problem and it sucks, and what ends up happening is you end up again porting that data from those disparate SQL and NoSQL databases either to a warehouse which is RDBMS or you put it into a datalake. That's again possible if you somehow manage to find all the drivers. You're doubly screwed if you have a hybrid - cloud and on-prem setup.

>> No one has solved that problem (...) you end up again porting that data from those disparate SQL and NoSQL databases either to a warehouse That's exactly how that problem has been solved successfully for the past 20 years.

The high latency between operational data and that data being reflected in reporting using traditional data warehouse pipelines makes it difficult for companies to make effective business decisions in a fast-paced business environment. Even in competent execution, that latency is frequently measured in weeks for big businesses. In the last few years, I've been approached by a number of traditional big businesses looking to rearchitect their database systems to make them more monolithic for the express purpose of reducing end-to-end latency in support of operational decisions.

It is extremely expensive and slow to move all the data to a data warehouse. Ignoring the cost element, the latency from when data shows up in the operational environment to when it is reflected in the output of the data warehouse is often unacceptably high for many business use cases. A large percentage of that latency is attributable solely to what is essentially complex data motion between systems.

Re: Monoliths Are the Future

#128
post #10

The author does not seem to understand when to correctly apply microservices. There are two basic use cases: 1) Different parts of your solution have different load patterns and it is economically beneficial to scale them at different rates and 2) Different teams need to be able to work & ship autonomously. It's not at all about technical merits or architectural beauty. It's about people and costs.

I kind of think the items (1) and (2) you list don't automatically mean micro-services, so much as they mean separation of concerns can be beneficial. Isn't there room for a middle ground with modularity that can live in between a full blown monolith or a full blown microservices pattern, particularly for operations that are more medium scale?

megaservices

Re: Monoliths Are the Future

#129

Earlier quoted context omitted.

But I’d argue Monoliths don’t have anything inherent to them which makes reporting easier. A proper BI setup requires a lot of hard work no matter how the backend services are built.

> But I’d argue Monoliths don’t have anything inherent to them which makes reporting easier. It's easier to join tables in databases that live on a single server, in a single database platform, than it is to connect to lots of different data sources that live in different servers, possibly even in different locations (like cloud vs on-premises, and hybrids.)

> It's easier to join tables in databases that live on a single server, in a single database platform

1) what if your data set is larger than you can practically handle in a single DB instance?

2) nothing about a monolith implies you have a single data platform, let alone a single DB instance

Re: Monoliths Are the Future

#130

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

It's a little sad because originally, people thought there would be a shared data base (now one word) for the whole organization. Data administrators would write rules for the data as a whole and keep applications in line so that they operated on that data base appropriately. A lot of DBMS features are meant to support this concept of shared use by diverse applications.

What ended up happening is each application uses its own database, nobody offered applications that could be configured to an existing data base, and all of our data is in silos.

Post reply on HN