Monoliths Are the Future • Hacker News Board

Re: Monoliths Are the Future

#351

Post by nitwit005 » Thu, Jan 30, 2020, 11:20 PM UTC

BrentOzar wrote: post #8

I'm a database guy, so the question I get from clients is, "We're thinking about breaking up our monolith into a bunch of microservices, and we want to use best-of-breed persistence layers for each microservice. Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting?" Analysts expect to be able to connect to one system, see their data, and write queries for it. They were neve…

You can't entirely escape this problem. Even when companies want to fit everything in a single database, they often find they can't. With enough data you'll eventually run into scalability limitations.

A quick fix might be to split different customers onto different databases, which doesn't require too many changes to the app. But now you're stuck building tools to pull from different databases to generate reports, even though you have a monolithic code base.

Re: Monoliths Are the Future

#352

Post by NeoBasilisk » Thu, Jan 30, 2020, 11:26 PM UTC

Ok, I read 5 years ago that microservices are the future and that monoliths are dead. I'm glad we're making progress.

Re: Monoliths Are the Future

#353

Post by qaq » Thu, Jan 30, 2020, 11:26 PM UTC

You move to microservices now you are in the land of distributed systems so unless you are at a scale that leaves you no choice be very aware of this. You are multiplying edge cases by a factor of 100 in many cases.

Re: Monoliths Are the Future

#354

Post by rjpn » Thu, Jan 30, 2020, 11:35 PM UTC

It depends a lot on the app your building. If you are a startup, it definitely is much easier to build a monolith and focus on the product features. Micro-Services may not have that much upfront cost of building but as your product grows it requires lot of engineering effort and budget to maintain it. You can have an engineering team of 20 people maintaining a monolith which can serve few million customers. The same product broken into micro-services will require 4-5 teams of 8 people. It is much easier to hire for a single skillset and grow the team.

Re: Monoliths Are the Future

#355

Post by StavrosK » Thu, Jan 30, 2020, 11:39 PM UTC

throwaway894345 wrote: post #323

StavrosK wrote: post #273

Earlier quoted context omitted.

I disagree, "doing it wrong" just looks different there.

The argument (which I sort of buy) is that microservices provide rails that keep people from doing certain stupid things like N clients depending on the data schema (making the schema a de-facto public interface). The trick with microservices is that the ecosystem is maturing and there are still lots of ways to screw up other things that are harder to screw up with monoliths. In time 95% of those will go away (my spe…

You basically just described Erlang/OTP there.

Re: Monoliths Are the Future

#356

Post by edoloughlin » Thu, Jan 30, 2020, 11:41 PM UTC

simonw wrote: post #331

MadWombat wrote: post #131

Earlier quoted context omitted.

> Some data belongs in Postgres, some in DynamoDB, some in JSON files. Now, how do we do reporting? One of the key concepts in microservice architecture is data sovereignity. It doesn't matter how/where the data is stored. The only thing that cares about the details of the data storage is the service itself. If you need some data the service operates on for reporting purposes, make an API that gets you this data and…

A co-worker had a smart solution for this: your service's representation in a reporting system (a data warehouse for example) is part of its API. Your team should document it, and should ensure that when that representation changes information about the changes is available to the people who need to know it. This really makes sense to me. I love the idea that part of a microservice team's responsibility is ensuring t…

your service's representation in a reporting system

At what point in time?

Re: Monoliths Are the Future

#357

Post by klez » Thu, Jan 30, 2020, 11:41 PM UTC

rohan-ramchand wrote: post #324

pmoleri wrote: post #308

Earlier quoted context omitted.

That's what semver is for, right? Breaking changes go in a major, same major means that the latest is always compatible. You'll have the same issue with microservices if you introduce breaking changes.

Yeah, but even with semver library owners have problems. If you need to make a critical change (e.g. a security update), you can either (a) wait until everyone does a version bump on their own schedule, or (b) do the version bump yourself, which means you're deploying every app that relies on your library. (a) might not be feasible for critical things, and (b) means that you might be deploying changes that the app is…

> For example, if master is v1.5.0, and I'm an app that uses v1.1.0, then if the library owner bumps to 1.5.1 for a critical update, I need to go from 1.1.0 -> 1.5.1, which might involve changes I'm not ready for yet. I better have phenomenal integration testing to make sure the update is safe to do.

That could be solved by backporting security fixes to version 1.1.

Of course at a certain point you should deprecate older versions of your package. At which point it's the client's responsibility to upgrade (like for any third party library they would be using).

Re: Monoliths Are the Future

#358

Post by james_s_tayler » Thu, Jan 30, 2020, 11:41 PM UTC

mewpmewp2 wrote: post #184

echelon wrote: post #165

Earlier quoted context omitted.

Humans are the most expensive part of the system. You have to make it easy for humans to understand and change the system, and at the end of the day that's the number one thing to optimize for. This is why microservices are compelling. But to speak directly to your concern, you have to think about service boundaries and granularity correctly. Nobody is saying make a microservice out of every conceivable table. Think…

In my experience it is a lot more difficult to navigate around all the different microservices to understand what needs to be done compared to being in a monolith where you can jump from file to file. Also then what also happens is microservices are created using different languages which in turn adds so much complexity to understand what is going on on the whole big picture level. And code gets repeated a lot more.…

>And code gets repeated a lot more. If there is change in a microservices or update everyone will need to figure out what services depend on and how they will have to adapt. With monolith you can just use your IDE to see what will break if you make a change. So much repeated business logic. Creating a new feature involves having to have many meetings to figure out what services in which way have to be updated.

This is what people mean when they say "distributed monolith" vs. microservices.

Re: Monoliths Are the Future

#359

Post by joefkelley » Thu, Jan 30, 2020, 11:45 PM UTC

dmitrig01 wrote: post #245

joefkelley wrote: post #212

Earlier quoted context omitted.

It's been about 4 years since I've been in this world, but I remember there being several products all doing a very similar thing: Presto, Hive, SparkSQL, Impala, perhaps some more I'm forgetting. Is the situation still the same? Or has Presto "won out" in any sense?

Hive and Impala are databases. Presto and SparkSQL are SQL interfaces to many different datasources, including Hive and Impala, but also any SQL database such as Postgres/Redis/etc, and many other types of databases, such as Cassandra and Redis; the SQL tools can query all these different types of databases with a unified SQL interface, and even do joins across them. The difference between Presto and SparkSQL is that…

I think some of this might be misinformed.

I know Hive can definitely query other datasources like traditional SQL databases, redis, cassandra, hbase, elasticsearch, etc, etc. I thought Impala had some bit of support for this as well, though I'm less familiar with it.

And SparkSQL can be run on a multi-tenant cluster with automatic resource allocation - Mesos, YARN, or Kubernetes.

Re: Monoliths Are the Future

#360

Post by andrewprock » Thu, Jan 30, 2020, 11:45 PM UTC

stock_toaster wrote: post #224

closeparen wrote: post #57

My employer adopted microservices for a very specific reason: it became nearly impossible to deploy the monolith. With hundreds of commits trying to go out every day, probability that at least one would break something approached 1. Then everything had to be rolled back. Getting unrelated concerns into separate deployable artifacts rescued our velocity. It came with many of its own challenges, too! A great deal of in…

> With hundreds of commits trying to go out every day, probability that at least one would break something approached 1. Then everything had to be rolled back. Getting unrelated concerns into separate deployable artifacts rescued our velocity. Software tends to reflect the structure of the organization that creates it, so this makes sense to me. If you have multiple teams contributing to a stack, eventually it is eas…

I think you are missing the intermediate step of libraries.

1. monolith 2. libraries 3. services

If you skip step 2, there is a high probability that the services you end up with are going to be just as disorganized as the monolith which is causing grief.