Monoliths Are the Future • Hacker News Board

Re: Monoliths Are the Future

#531

Post by zzzeek » Fri, Jan 31, 2020, 5:03 PM UTC

tacitusarc wrote: post #491

zzzeek wrote: post #408

Earlier quoted context omitted.

> Take for example Celery. With Celery all you need to do to implement a queuable background task system that's trivially scalable is to write the background tasks, that's a little bit of a straw man because that's not the "microservice" architecture this post is talking about. I personally wouldn't call that a "microservice" architecture, I'd call it, "a background queue", although strictly speaking it can be descri…

Programmers are, by and large, quite bad at not breaking encapsulation when they're dealing with a monolith. Not their fault, really, but when management comes to you and says, hey, can't you get it done in just a day, we really need this, and you know you could if you just hacked through that particular isolation barrier just this once , and yeah it will create bad bugs if things change in a particular way in the fu…

I'm really tired of immensely awkward and problematic design patterns that are intended to act as guard rails for good design. It is a myth that this actually works. A rushed project is a rushed project; a team that is inclined to overbuild will overbuild no matter what artificial constraints you give them up front. A microservice design that has concurrency problems can be extremely difficult to debug, in my personal experience this is easily much more difficult than debugging a monolith. Having to spin up 30 services in order to reproduce an issue that could otherwise be done in a single in-memory unit test is a real thing.

Re: Monoliths Are the Future

#532

Post by bkanber » Fri, Jan 31, 2020, 5:07 PM UTC

aliswe wrote: post #496

bkanber wrote: post #71

I've settled on a compromise in this debate. Halfway between monoliths and microservices is the shared-library model. Instead of creating a microservice for your image processing, break it out into a standalone NPM or Composer or whatever module, then use that in your monolith. Gives you good separation of code and responsibilities, gives you good upgrade paths for your monoliths, avoids the overhead of microservices…

Not trying to cargo cult, but what about the 12-factor stance that shared code is an anti-pattern?

Sorry, but I don't think that's part of the 12-factor app. The 12-factor guideline is that apps shouldn't be dependent on implicit dependencies (I dunno, like having imagemagic installed on the system), but instead should use a dependency management system (like NPM or Composer). So I don't think the statement "shared code is an anti-pattern" is true. But I could be wrong, haven't read the 12 factors in a while...

Re: Monoliths Are the Future

#533

Post by jy2947 » Fri, Jan 31, 2020, 5:11 PM UTC

I think we all want 3 things -

1, back-end services with clear boundary, that decouple concerns based on dev teams' domain responsibilities, with less dependency among each other,and respected source of record. This is very much the "micro-service" is for.

2, middle tier services to consolidate or aggregate back-end APIs to serve the front-ends (especially the mobile apps) and take care of the business logic. Back-end guys all love micro-services, but someone must put them all together....GraphQL so far seems to fit this bill

3, Analytics and reporting, this is a totally different animal from the product development, and have almost opposite requirements. This is where whatever your ETL or Data Lake or Data Pipeline is used, along with your preferred BI or analytics tooling.

Re: Monoliths Are the Future

#534

Post by pjmlp » Fri, Jan 31, 2020, 6:20 PM UTC

MockObject wrote: post #525

pjmlp wrote: post #484

Earlier quoted context omitted.

Have you ever used CORBA and DCOM factories? Enterprise ORMs were created in Smalltalk, C++ and Objective-C, years before Java was created. Poet and Enterprise Object Framework were two well known ones. In fact J2EE was born of the ashes of a failed Objective-C project at Sun, Neo, after its collaboration with NeXT on OpenSTEP failed apart. https://en.m.wikipedia.org/wiki/Distributed_Objects_Everywhe... Things are no…

Teaching the parable of Chesterton's Fence to every programmer would save billions annually. https://wiki.lesswrong.com/wiki/Chesterton%27s_Fence

Indeed.

Re: Monoliths Are the Future

#535

Post by rubyn00bie » Fri, Jan 31, 2020, 7:45 PM UTC

IggleSniggle wrote: post #483

galaxyLogic wrote: post #443

Earlier quoted context omitted.

Micro-services are just connected things which work together to accomplish something. But where are those connections described? In some tables somewhere. Maybe. Whereas if you write a single monolithic program its connections are described in code, preferably type-checked by a compiler. I think that gives you at least theoretically a better chance of understanding what are the things that connect, and how they conne…

There is such a language, and it is Erlang.

I write Elixir professionally, Erlang/Elixir and BEAM is no one-stop solution to these problems either. It has tools to help you, but you can very easily end up in the same boat.

Re: Monoliths Are the Future

#536

Post by shanemlk » Fri, Jan 31, 2020, 9:33 PM UTC

Yet another extremely vague Hacker News article building controversy due to its failure to communicate anything tangible. Is this article talking about monolith deployment architectures or monolith code bases?

If it's about monolith deployment architectures, the use case is really important.

If it's about monolith code bases, you need to define what a monolith code base even means, because that could mean anything. Are we talking storage size, custom written code lines, framework architecture, or just the number of people and/or teams building the underlying technology?

Re: Monoliths Are the Future

#537

Post by rubyn00bie » Fri, Jan 31, 2020, 10:07 PM UTC

mycall wrote: post #460

rubyn00bie wrote: post #347

I couldn't agree more with an article. Most people think a micro-service architecture is a panacea because "look at how simple X is," but it's not that simple. It's now a distributed system, and very likely, it's a the worst-of-the-worst a distributed monolith. Distributed system are hard, I know, I do it. Three signs you have a distributed monolith: 1. You're duplicating the tables (information), without transformin…

> Don't duplicate data without adding value to it. What about running multiple versions of a microservice in parallel -- don't each need their own yet separate databases that attempt to mirror each other as best they can?

I'm assuming you mean in production micro-service, if that's not the case, please elaborate a bit more...

The short answer is "no," as succinctly stated by the, I assume from the name, majestic SideburnsOfDoom. The versions shouldn't _EVER_ be incompatible with each other.

E.g. you need to rename a column.

Do not: rename the column, e.g. `ALTER TABLE RENAME COLUMN...`. Because, your systems are going to break with the new schema.

Do: Add a new column, with the new name, and migrate data to the new column, once it's good upgrade the rest of your instances, then drop the old column. Because, you can use both versions at the same time now without breaking anything. Yes, it can be a little tricky to get the data sync'd into the new column, but that's a lot less tricky than doing it for _every_ table and column.

Re: Monoliths Are the Future

#538

Post by sigotirandolas » Fri, Jan 31, 2020, 10:08 PM UTC

keanebean86 wrote: post #209

tynpeddler wrote: post #181

There are two big reasons to go to microservices (note that the exact definition of microservice can vary a lot). 1. Organizational streamlining. If the team working on the monolith becomes to large, then coordinating and pushing out changes quickly can become incredibly difficult. One rule of thumb I've heard is the two pizzas rule. If two pizzas can't feed the team working on a system, it's time to break up the sys…

I have this idea for a new framework/language. I'm sure if it either already exists or it's a dumb idea in practice but anyways. You build a monolithic application. Everyone works on the same code base. Things are broken up into modules/classes/packages. From the programmers point of view it's just like working on a standard Java project or something similar. The magic happens at the method and module boundaries. Whe…

I don't personally have much experience on this area, but something as valuable as this holy grail of automatic parallelization (be it threading or distribution), i.e. "just write the same good old code and the system will be smart and parallelize it" is bound to be something that people have been trying forever and failing, at least on the general case.

To begin with, anything with side effects is mostly a non-starter. You would need some way to annotate the boundaries where side effects can not happen, and this already requires considerable refactoring effort.

Even if you ignore this and suppose you are working with a purely functional language (this is probably what the "You just invented Erlang and OTP " meant), the overhead of the "smartness" can be pretty big. If you see a "map" over a list, you know you can parallelize it, but which one of the 100 "map"s should you really parallelize? If you try to be too smart, you will burn a lot of effort and may not get a payoff. If you try to be just a little big smart and you choose the wrong one you shuffle a lot of data for nothing.

In some way it seems to me a lot of today's big data systems like (surprise) MapReduce and Spark do exactly this, they offer ways to explicitly mark those boundaries where you want your program to be parallel (and require that you obey their rules about side-effects there) and contain some of this "smartness" for how to distribute your data.

Even in OpenMP you also have something like this, you can easily say "this piece of code is a task and the runtime can decide to run it in parallel", but you need to also tell it all the inputs, outputs, etc. (since figuring this out automatically is hard if not impossible) so it doesn't fit very well in a "big ball of mud" project.

In the (near?) future as computing power is more abundant and cheaper but the gap between sequential and parallel computing continues growing ever wider I can see those general "smart" approaches paying off, even if they are much worse than hand-optimized code. It only needs to be better than the average programmer.

Re: Monoliths Are the Future

#539

Post by zamalek » Fri, Jan 31, 2020, 10:09 PM UTC

smoyer wrote: post #406

rubyn00bie wrote: post #347

I couldn't agree more with an article. Most people think a micro-service architecture is a panacea because "look at how simple X is," but it's not that simple. It's now a distributed system, and very likely, it's a the worst-of-the-worst a distributed monolith. Distributed system are hard, I know, I do it. Three signs you have a distributed monolith: 1. You're duplicating the tables (information), without transformin…

We've moved a lot of services into Kubernetes and broken things up into smaller and smaller micro-services. It definitely eliminates a lot of the complexity for developers ... but you trade it for operational complexity (e.g. routing, security, mis-matched client/server versions, resiliency when dependency isn't responding). I still believe that overall software quality is higher with micro-services (our Swagger docu…

> It definitely eliminates a lot of the complexity for developers

We're currently translating a 20 year old ~50MLOC codebase into a distributed monolith (using a variety of approaches that all approximate strangler). I have far less motivation to go to work if I know that I will be buried in the old monorepo. I can change, build and get a service changed in less than an hour. Touching the monorepo is easily 1.5 days for a single change.

We seem to be gaining far more in terms of developer productivity than we are losing to operational overhead.

Re: Monoliths Are the Future

#540

Post by rubyn00bie » Fri, Jan 31, 2020, 10:33 PM UTC

rumanator wrote: post #504

zzzeek wrote: post #408

Earlier quoted context omitted.

> Take for example Celery. With Celery all you need to do to implement a queuable background task system that's trivially scalable is to write the background tasks, that's a little bit of a straw man because that's not the "microservice" architecture this post is talking about. I personally wouldn't call that a "microservice" architecture, I'd call it, "a background queue", although strictly speaking it can be descri…

> that's a little bit of a straw man because that's not the "microservice" architecture this post is talking about. I personally wouldn't call that a "microservice" architecture, I'd call it, "a background queue", although strictly speaking it can be described as such. It is not a strawman; it's a concrete example of the technical, practical, operational, and economical advantages of microservices architecture, more…

> While you're groking how a multihreadig library is expected to be used, I already fired a message broker that distributes tasks across a pool of worker instances. Why? Because I've opted not to go with the monolith and went with the microservices/distributed architecture approach.

I'm so confused by that statement... because I can't for the life of me figure out how you got there.

You absolutely can have a monolith which is multi-threaded or asynchronous and "resource/task pools." The JVM for instance has threads, I use BEAM (Elixir) personally, and it's even pre-emptively scheduling my tasks in parallel and asynchronously... but, I still don't get what multi-threading has to do with microservices.

Microservices and monoliths are boundaries for your application they aren't implementation details (i.e. all microservices must be asynchronous is strictly not true) in and of themselves, they're design details. That design can influence the implementation but they are separate.

Ex. there are plenty of people who use Sidekiq and redis like you're using Celery but don't call it a microservice. It's just a piece of their monolith since it's largely the same depdencies.