Monoliths Are the Future • Hacker News Board

Re: Monoliths Are the Future

#341

Post by cc81 » Thu, Jan 30, 2020, 10:56 PM UTC

notJim wrote: post #233

I dunno if I'm taking crazy pills, but it seems like the organizational/deployment-level concerns supposedly solved by microservices are much better solved by continuous deployment, and it's weird to me that this seems to be such a contrarian viewpoint. Continuous deployment alleviates merge/coordination issues by integrating small changes frequently, which makes conflicts rare. Deploys are safer, again because you'r…

I think it is about autonomy and planning as well. You own your API, you plan your features and your backlog will be filled and you will deliver it. You will sunset it and when you on-board new people into the team they only need to learn your part.

This requires quite a big application though for it to be worth it in my opinion.

Re: Monoliths Are the Future

#342

Post by jeffnappi » Thu, Jan 30, 2020, 10:57 PM UTC

I'm happy to see this article on the front page it certainly resonates with my perspective - however I'm not sure it's really an either-or situation.

One thing I've noticed is that big tech is taking advantage of giant mono-repos, while everyone else is stuck with 10s-100s of git repositories haphazardly connected and managed. For example - most off-the-shelf CI systems and VCS platforms smaller organizations are using are per-repository (GH, GH Issues, CircleCI, etc).

Managing micro-services would be a far easier task when all of the services (and infrastructure as code) live in the same repository, changes can be staged across multiple services at once, and tests are automatically ran for only the necessary dependencies.

Are there solutions for effective mono-repo management outside of FAANG? Am I wrong? :)

Re: Monoliths Are the Future

#343

Post by PopeDotNinja » Thu, Jan 30, 2020, 10:59 PM UTC

eternalny1 wrote: post #35

What is old is new again. I've been a software engineer for over 30 years and have dealt with companies always trying to jump on the next bandwagon. One company I worked with tried to move our entire monolith application, which was well architected and worked fine, over to a microservices-based architecture and the result was an unstable, complex mess. Sometimes, if it's not broke, don't try to "fix" it. I can say th…

> Sometimes, if it's not broke, don't try to "fix" it.

I'm going to interpret this as "if your monolith is broke, breaking it up into microservices won't fix it"

Re: Monoliths Are the Future

#344

Post by debrice » Thu, Jan 30, 2020, 11:04 PM UTC

abhishekjha wrote: post #300

debrice wrote: post #297

Earlier quoted context omitted.

Where I work we use micro-services through lambda (we have dozens of them) and use DynamoDB for our tables. DynamoDB streams are piped through elasticsearch. We use it for our business intelligence. Took us about a week to setup proper replication and sharding. I don't have a strong opinion on monolith or micro-service, pick one or the other, understand their culprit and write high quality (aka. simple and maintainab…

Is there a way to do this for a relational database at scale?

We used to do that but dynamodb being very dynamic (which we like) makes it harder to use a declarative schema. ES auto detect the schema which makes it a breeze. I’d say try with JSON fields but I don’t think you’ll get great query speed

Re: Monoliths Are the Future

#345

Post by nitwit005 » Thu, Jan 30, 2020, 11:05 PM UTC

pmoleri wrote: post #308

nitwit005 wrote: post #290

Earlier quoted context omitted.

My previous company went this direction. I wouldn't recommend it. Say you version each module and pull in specified versions. It'll work fine, right up until two modules both try to pull different versions of a third module. In practice, you have to update multiple modules at once to avoid conflicts, which, in turn, can require updating other team's code. It also turns out some tools like Maven don't prevent conflict…

That's what semver is for, right? Breaking changes go in a major, same major means that the latest is always compatible. You'll have the same issue with microservices if you introduce breaking changes.

That would have let you know when to expect the impact, but not eliminate the impact itself.

Occasionally people versioned their module's APIs, which seemed like a cleaner way to handle the module update, as you don't have to update everything at once. They only went to that effort when they realized they'd have to update thousands of callers.

Re: Monoliths Are the Future

#346

Post by etxm » Thu, Jan 30, 2020, 11:05 PM UTC

abhishekjha wrote: post #300

debrice wrote: post #297

Earlier quoted context omitted.

Where I work we use micro-services through lambda (we have dozens of them) and use DynamoDB for our tables. DynamoDB streams are piped through elasticsearch. We use it for our business intelligence. Took us about a week to setup proper replication and sharding. I don't have a strong opinion on monolith or micro-service, pick one or the other, understand their culprit and write high quality (aka. simple and maintainab…

Is there a way to do this for a relational database at scale?

You can do change data capture with Debezium.

Recently the Netflix engineering blog mentioned a tool called DBLog too, but I don’t believe they’ve released it yet.

Re: Monoliths Are the Future

#347

Post by rubyn00bie » Thu, Jan 30, 2020, 11:06 PM UTC

I couldn't agree more with an article.

Most people think a micro-service architecture is a panacea because "look at how simple X is," but it's not that simple. It's now a distributed system, and very likely, it's a the worst-of-the-worst a distributed monolith. Distributed system are hard, I know, I do it.

Three signs you have a distributed monolith:

1. You're duplicating the tables (information), without transforming the data into something new (adding information), in another database (e.g. worst cache ever, enjoy the split-brain). [1]

2. Service X does not work without Y or Z, and/or you have no strategy for how to deal with one of them going down.

2.5 Bonus, there is likely no way to meaningfully decouple the services. Service X can be "tolerant" of service Y's failure, but it cannot ever function without service Y.

3. You push all your data over an event-bus to keep your services "in-sync" with each-other taking a hot shit on the idea of a "transaction." The event-bus over time pushes your data further out of sync, making you think you need an even better event bus... You need transactions and (clicks over to the Jepsen series and laughs) good luck rolling that on your own...

I'm not saying service oriented architectures are bad, I'm not saying services are bad, they're absolutely not. They're a tool for a job, and one that comes with a lot of foot guns and pitfalls. Many of which people are not prepared for when they ship that first micro service.

I didn't even touch on the additional infrastructure and testing burden that a fleet of micro-services bring about.

[1] Simple tip: Don't duplicate data without adding value to it. Just don't.

Re: Monoliths Are the Future

#348

Post by toriningen » Thu, Jan 30, 2020, 11:11 PM UTC

BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

Or you could have CQRS projectors (read models), which solve exactly this - they aggregate data from lots of different eventually consistent sources, providing you with locally consistent view only of events you might be interested in.

It will lag behind by some extent, roughly equal to the processing delay + double network delay, but can include arbitrary things that are part of your event model.

Though, it's not a silver bullet (distributed constraints are pain in the ass yet), and if system wasn't designed as DDD/CQRS system from the ground up, it would be hard to migrate it, especially because you can't make small steps toward it.

Re: Monoliths Are the Future

#349

Post by UglyToad » Thu, Jan 30, 2020, 11:16 PM UTC

MosheZada wrote: post #43

Why do we need to choose one of monolith and microservices? What about simply "services"? Monolith doesn't have to be split into 50 microservices, it can be split to 3 services

Yeah, I don't have much experience with microservices but the one time I worked with them they were a nightmare and I think the major cause was there were too bloody many of them.

I feel like if you end up talking about eventual consistency or needing ids to be passed around you've surely built it wrong and you've split what should be a single service into a mess just to feel cool having more services and needing the hyped new tools to manage them.

The touted benefit that you can scale the bottleneck separately also suggest the need for 1 monolith at most 1 or 2 services. If you have more than that many bottlenecks the code is doomed anyway presumably?

Re: Monoliths Are the Future

#350

Post by jrochkind1 » Thu, Jan 30, 2020, 11:17 PM UTC

tootie wrote: post #220

closeparen wrote: post #57

My employer adopted microservices for a very specific reason: it became nearly impossible to deploy the monolith. With hundreds of commits trying to go out every day, probability that at least one would break something approached 1. Then everything had to be rolled back. Getting unrelated concerns into separate deployable artifacts rescued our velocity. It came with many of its own challenges, too! A great deal of in…

The solution to this is just writing modular code and using an artifact repository. It's a model I've rarely seen attempted even though it's much easier than microservices and serves the same purpose. You can have individual dev teams, with their own repo ,backlogs, own stakeholders, etc all working at their own paces. They build modules (jars, nuget packages, npm modules) and deploy semver versioned artifacts to a r…

Interesting. Ie, just the way the larger community (ie open source) ecosystem works, but just inside your company instead.

That makes an obvious kind of sense, when your number of contributors have grown too large to operate like a monolith... why not operate using models well-established for very large inter-entity communities?

I wonder why more very large companies don't do this, if they don't.