Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monoliths Are the Future

changelog.com

Post reply on HN
Print view View original

91–100 of 567 posts

Re: Monoliths Are the Future

#91

Post by benwaffle »

JMTQp8lwXL wrote: post #79
BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

Whether or not your system intentionally ended up with this architecture, GraphQL provides a unified way of fetching the data. You'll still have to implement the details of how to fetch from the various services, but it gives people who just want read access to everything a clean abstraction for doing so.

REST can do the exact same thing.

Re: Monoliths Are the Future

#92

Post by skywhopper »

rjkennedy98 wrote: post #19
BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

This is what Kafka is for. You put Kafka on top of your database to expose data and events. Now BI can take the events put them into their system as they want.

Worth pointing out that running Kafka is itself no small thing. So now you've added BI and also Kafka, which itself requires a bundle of services to operate. And you have to keep BI and Kafka in sync with all your various other data stores' schemas, and with each other.

Which all gets back to the point of the OP.

Re: Monoliths Are the Future

#93

Post by thrower123 »

tylerjwilk00 wrote: post #87
thrower123 wrote: post #58

Earlier quoted context omitted.

I can't imagine your level of cynicism. I've only been at this for ten years, and the number of times I've seen the wheel come full circle and old ideas come back into vogue, the problems with them rediscovered, reactions to those problems, and then the thing that preceded them again take precedence is somewhat depressing. At best I feel like we are grinding ahead a few inches each cycle.

It's like the movie Benjamin Button but with a case of amnesia every three years.

Sometimes it feels more like Memento

Re: Monoliths Are the Future

#94

Post by jandrewrogers »

gowld wrote: post #26
BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

Build a reporting database that maintains a copy of data from other data stores. Simple enough. Surely you wouldn't run analytics directly on your prod serving database, and risk a bad query taking down your whole system?

There are many business environments where copying the data is effectively intractable so you need to run all of your data model operations in a single instance. More data models have this property with each passing year, largely because copying data is very slow and very expensive.

This is not a big deal if your system is designed for it, sophisticated database engines have good control mechanisms for ensuring that heavy reporting or analytical jobs minimally impact concurrent operational workload.

Re: Monoliths Are the Future

#95

Post by pjmlp »

MockObject wrote: post #65
eternalny1 wrote: post #35

What is old is new again. I've been a software engineer for over 30 years and have dealt with companies always trying to jump on the next bandwagon. One company I worked with tried to move our entire monolith application, which was well architected and worked fine, over to a microservices-based architecture and the result was an unstable, complex mess. Sometimes, if it's not broke, don't try to "fix" it. I can say th…

It used to be that people wrote Fortran in any language. Now they're rewriting Java in every language.

The bad reputation that enterprise Java gets, comes from enterprise C and C++ codebases.

https://en.wikipedia.org/wiki/Structured_systems_analysis_an...

https://en.wikipedia.org/wiki/Booch_method

Any language that gets into enterprise architect hands, with projects spread around multiple development sites with several consulting agencies, gets their FactoryFactories and such.

Re: Monoliths Are the Future

#96

Post by rezeroed »

DanielBMarkham wrote: post #31

With respect to the author, who probably is a much smarter person than I am, this is yet another in a long, long series of HN articles that should be grouped under "I don't know what the hell X is, but I was an expert in it, and I can tell you it sucks" I've seen X be a dozen things: UML, databases, User Stories, Functional Programming, Testing... It's too much to list. Yes. If you do it that way it will hurt, and yo…

"who probably is a much smarter person than I am" -- I have a different take-away from the quality of that writing. I'm also no fan of the kubernetes code.

Re: Monoliths Are the Future

#98

Post by jsjohnst »

BrentOzar wrote: post #48
rjkennedy98 wrote: post #19

Earlier quoted context omitted.

This is what Kafka is for. You put Kafka on top of your database to expose data and events. Now BI can take the events put them into their system as they want.

> You put Kafka on top of your database to expose data and events. Now BI can take the events put them into their system as they want. Right, that's the data warehouse method that I described. "Put them into their system" is a lot harder work than just typing in "put Kafka on top of your database."

Things like Maxwell[0] help a lot with the getting stuff into Kafka from the DB, but I agree with your point entirely. Kafka is not something I’d recommend unless you’ve got a team dedicated to maintaining it. Thankfully Confluent is rising up to fill that need and reasonably priced. Confluent also has an offering for streaming binlogs into Kafka too.

[0] http://maxwells-daemon.io/

Re: Monoliths Are the Future

#99

Post by js8 »

gowld wrote: post #26
BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

Build a reporting database that maintains a copy of data from other data stores. Simple enough. Surely you wouldn't run analytics directly on your prod serving database, and risk a bad query taking down your whole system?

That's the data warehouse option.

Re: Monoliths Are the Future

#100

Post by zten »

BrentOzar wrote: post #74
zten wrote: post #27

Earlier quoted context omitted.

I don't think you're right back to a monolith with centralized reporting. Remember, microservices doesn't mean JSON-RPC over HTTP. Passing updates extracted via change data capture and forwarding them to another reporting system is a perfectly viable interface. Data duplication is also an acceptable consequence in this design.

> Passing updates extracted via change data capture and forwarding them to another reporting system is a perfectly viable interface. Right, that's the data warehouse method I described, keeping a central database in a reporting system. But now you just have to keep that database schema stable, because folks are going to write reports against it. It's a monolith for reporting, and its schema is going to be affected by…

Sorry, I noticed you made the same observation in another thread after you got dogpiled by everyone; I left my question over there. Yeah, I've had to contend with that problem before -- hard to imagine how I forgot about it :)
Post reply on HN