Monolith First (2015)

That’s what I am always saying. If you can’t manage libraries you will fail at microservices too, just harder.

I am working on a project right now that was designed as microservices from the start. It’s really hard to change design when every change impacts several independent components. It seems to me that microservices are good for very stable and well understood use cases but evolving a design with microservices can be painful.

I, like many others, have been saying this for years. Too bad I didn't see this link before, it would have helped convincing a few people in the past. Now, fortunately, it doesn't seem that much of a heresy to say that monoliths (or SOA) is the right architecture a lot of the time (maybe most of the time).

I remember, a little more than 4 years ago I was brought on to save a product launch for a startup. They were, as far as I can remember, already creating the 2nd iteration/rewrite of their MVP with the first one never released and just few months short of doing their first release one of their developers started to convince everyone that the code base was a pile of crap and that they need to rebuild the whole thing is microservices. Because there is no other way around. The system was built in PHP and the otherwise smart and motivated developer wanted to rebuild it as a set of JS microservices.

He almost managed to convince the management, including the devlopment lead and the rest of the team. And it wasn't easy to convince him that that would have been a pretty dumb move and that it wouldn't have just taken a few more months, just because creating a 'user service' with a mongo backend was something he could pull of in his spare time.

Then I realized that there is something inherent in these kind of situations. Some people come around stating that a new piece of technology (or other knowledge) is better and then suddenly the rest of the world is somehow on the defense. Because they know something the rest don't so how could you prove them wrong? Not easy. And funny enough, this is actually a classic case of the shifting of the burden of proof fallacy.

The question is from what team size on microservices make things easier. I bet the number is much bigger than what people think.

Yeah, that's a terrible idea, so don't do something like that. You don't need to break up your monolith to run cronjobs.

My setup: I have async worker running alongside my application. Same codebase, same environment, same monolith, just instead of accepting HTTP requests it pops tasks off a queue. The framework handles retrying/reprocessing/failure.

To run cron tasks, I have CloudWatch Events -> SNS -> HTTP endpoint in the monolith which pushes the cron task into the queue.

And this isn't some unusual setup. This is something that people have been dealing with for a very long time, long before Lambda came into existence. Most web frameworks have async tasks built-in or an integration with a framework that can do async tasks.

How do you individually deploy and operate modules, packages, and libraries? The purported benefits of micro services have always been harmonizing with Conway's law. In particular, one of the nice benefits of microservices is that it's harder to "cheat" on the architecture because it likely involves changing things in multiple repos and getting signoff from another team (because again, the code organization resembles the org chart). The high-level architecture is more rigid with all of the pros and cons entailed.

I suppose if you work in an organization where everyone (including management) is very disciplined and respects the high level architecture then this isn't much of a benefit, but I've never had the pleasure of working in such an organization.

That said, I hear all the time about people making a mess with micro services, so I'm sure there's another side, and I haven't managed to figure out yet why these other experiences don't match my own. I've mostly seen micro service architecture as an improvement to my experiences with monoliths (in particular, it seems like it's really hard to do many small, frequent releases with monoliths because the releases inevitably involving collaborating across every team). Maybe I've just never been part of an organization that has done monoliths well.