Monorepo Support

> However, separating a software project into multiple repositories with their own dependencies, deployments, tests, and philosophies adds needless overhead and complexity to systems

Yes, for the organization. However when the organization is larger, this may not be a disadvantage - if the teams are already as large as small startups, then it only makes sense that they have their own repo if they are doing microservices.

They drift. True. But that's no different from the dependencies in a repo that comes with reusing code. Its good practice, but it also has its drawbacks.

> A solution to this is to just make [functionality X_new] and use some if statements

That's a worse practice in the long run. Such exceptions and slightly modified functions complicate the codebase and make it more difficult to gain keep context for anyone working on that codebase in the long run. There are situations in which this is inevitable. But if it can be avoided, it should be avoided.

> You can set up integration tests that prevent you from deploying if you've broken compatibility

Nope. Trust me, you eventually can't. Things will get complicated in the long run. You wont be able to have tests for every important angle, use case or function and maintain it. User-facing interfaces and functions are even more difficult - they involve combining all of those different services and functionality in a coherent whole. Move one brick and everything will get disrupted. You can try. But your tests, your commits, deploys will take much longer and everything will get more complicated.

> This more or less applies to any part of any software system unless you're very careful about shared state and side effects

Yes it does. Microservices is a way of avoiding that as long as possible. Eventually testing will still get complicated in user-facing functionality. But until your app becomes such a large and well-featured and used one, you have pretty good runway with microservices.

> I tend to think code duplication indicates you need to reconceive the mental model of your application, not merely create a helper method or whatever.

You can do that at the start. And you will be able to do it for a good chunk of time. But when your application is large and complicated enough, it will become more difficult to do. Microservices is a way to keep things isolated and contexts understandable as long as its possible.

> they lead to weird, rickety systems that are hard to develop and maintain. You haven't lived until your tickets for a sprint involve fixing multiple bugs in code copy-pasted across half a dozen microservices

Yes, that is an inherent difficulty in microservices. Tracking bugs, logging must improve. They eventually will.

> or you have to deploy your changes to this microservice you've never worked on before and it involves the most bonkers incantations you've ever read about (update this Jenkins script blah blah).

That is not specific to microservices. It can easily be encountered when dealing with a service that is tightly integrated in a monorepo, or even the part of a singular monolithic app.

Simple standards must be applied to all code across all microservices for keeping them simple, easily understandable and modifiable.

> "1 repo per project"

Doesn't that converge to the microservice model...

> we should all actively be fighting against Conway's law

I think the inverse of that, the org mirroring the code, happens as well. And maybe Conway really meant that, too.

I would argue that we should be organizing code in a way that we'd like our teams to be organized. We should use this as a tool for devs to self-organize. Eventually, management will see the cost savings in organizing the people similarly.

> You could setup your CI to only recompile what's changed, so you wouldn't be waiting for anything else than what you've changed.

What’s it like having CI pipelines that aren’t trash? I’m stuck with the awful choice of CI tool my work has made, and there’s not a lot I can do about that.

Of course, but how do you solve that problem in the general case?

When everything lives next to everything else it's harder to draw the line around the blast radius of a change. If there was a repo boundary, that's a really easy hard constraint.

> Yes, for the organization. However when the organization is larger, this may not be a disadvantage - if the teams are already as large as small startups, then it only makes sense that they have their own repo if they are doing microservices.

This is a pretty good point. I've not dealt with microservices at a huge company like a Netflix or a Wal-Mart. My experience is "we have a website with 10-20 pages, built by 1-3 teams of ~5 on 40 microservices". Maybe they make sense when you're at the scale of 100s of engineers, but they're not better than Rails/Django for 99% of companies.

> That's a worse practice in the long run. Such exceptions and slightly modified functions complicate the codebase and make it more difficult to gain keep context for anyone working on that codebase in the long run.

Oh, I was saying there's an easier solution than forking a repo. My overall point is that adding new functionality is a core software engineering thing; we should see it coming, and architect our code to--reasonably--accommodate it. To that end, sometimes we have to refactor stuff, which means leaning on your {integration,unit} tests. You're gonna have tests anyway, why not let them help you maintain compatibility?

> You wont be able to have tests for every important angle, use case or function and maintain it.

A lot of mission-critical industries (aerospace, transit, medical devices) more or less achieve this. It comes at a velocity cost, but it's not impossible. I'm just gonna hand wave and say you can probably get 80% of the effect with 20% of the effort, which is great. The point isn't to catch all bugs every time, the point is to let you modify functions/etc. without forking a new repo. And again, you're gonna have tests anyway.

>> This more or less applies to any part of any software system unless you're very careful about shared state and side effects

> Yes it does. Microservices is a way of avoiding that as long as possible.

Well, another way of saying this is "microservices have this problem too, just later". Sometimes that's useful, but if the agreement we're forging out here is "microservices are useful if you have a big company with lots of services/teams", it sounds like you'll inevitably have this problem. The solution, therefore, isn't microservices, it's managing shared state and side effects, which microservices doesn't have a monopoly on by any stretch.

> You can do that at the start. And you will be able to do it for a good chunk of time. But when your application is large and complicated enough, it will become more difficult to do.

This makes me think we disagree a little about what a microservice is--in fairness it's kind of a vague term. The microservices I've dealt with are like, a small web app in some micro framework (Node, Flask, Tornado, Go) that gets iframe'd into a website along with other microservices. My problem with this is: this should be a single website built in a single framework. Django, Rails, Phoenix, Symfony, etc. are all great at this, or you can get some backend-as-a-service like Hasura or PostgREST if you're willing to use minimally fancy JavaScript.

People will argue, "but $FRAMEWORK gives you no tools for managing shared state and side effects". But it does: the database. And I would also argue that having a single project lets you consolidate the implementation of your business logic. Using microservices, your business logic is copy-pasted across dozens of services. That's pretty clearly bad--wouldn't it be better if there were something like a libcompany that everyone shared?

You'd probably argue that it'd be hard to change libcompany and that every team should have lots of forks of libcompany that implement whatever bespoke changes they need so they can move fast. But that sounds nightmarish to me, especially as someone who's debugged this kind of situation before. And guess what, when you find the problem, you might not even get to fix it because microservices still have dependencies, some of which are unknown, so you lose that benefit also. What you'll probably do is fork another microservice with your libcompany changes.

>> or you have to deploy your changes to this microservice you've never worked on before and it involves the most bonkers incantations you've ever read about (update this Jenkins script blah blah).

> That is not specific to microservices.

It kind of is though, in that with "1 repo per project" you only have 1 weird pipeline to deal with. I can manage that. I find it hard to manage a dozen weird pipelines, or dozens of weird pipelines on different old versions of deployment/testing tools or custom scripts, etc. Maybe the vision of microservices is a fleet of Lambdas carefully managed by Terraform. That sounds nice! My experience with microservices is a hot mess of everything from Chef to Make to Jenkins. Scale matters here: dealing with 1 Chef and 1 Make is much better than dealing with 4 Chefs, 3 Makes, and 5 Jenkins.

>> "1 repo per project"

> Doesn't that converge to the microservice model...

Depends on what you mean by project I suppose. I'm not a fan of Domain-Driven Design, but one of the things I do like about it is how it defines a domain, which is a namespace without clashes. For example take the word "job". In the construction.com domain, "job" might mean the building you're building, which has certain attributes: building address, crew size, etc. In the softwareconsultant.com job domain, "job" might mean a contract you have with a client, so client name, requirements, etc. To me, these should be separate projects--two Rails codebases with two databases, testing setups, deployment pipelines, etc.

I don't think there's any convergence to the microservice model here. Microservices say, "what if construction.com was actually multiple Sinatra apps". Unless you're very into the microservice ideology, it sounds like these projects should stay single projects.

---

Leaving out all the "bad" implementations of microservices (which I think the microservices model makes really easy to do), I still think the "fleet of Lambdas managed by infra-as-code" leads to situations where you have copy-pasted versions of your business logic everywhere. This is solvable with a "libcompany", but microservices proponents are against that because it necessitates "coupling" and "coordination". But that's what a software system is: a bunch of interacting parts. I often find myself lost in a semantic graveyard in these debates because it's like, what is a project, is each microservice a project or is the project now defined in terms of dozens of microservices, blah blah blah. It's hard for me to not see these systems as simply expensive, complicated ways to deploy individual API endpoints with the attendant overhead of multiple deployment/maintenance/testing setups. Maybe that's useful for some organizations, like if construction.com is a spawling billion dollar enterprise with tons of complicated functionality. That sounds like a hard problem. But I think most people don't have hard problems, they don't need Martin Fowler, they just need Django or Hasura.