Live data from Hacker News

Keep the monolith, but split the workloads

incident.io

11–20 of 161 posts

Re: Keep the monolith, but split the workloads

#11
One critique I have is that this presents a binary option of either full monolith and microservices.

The truth is somewhere in between where you have dependent services large enough to warrant being their own monolith being split off. Breaking up of a monolith is almost never (anecdotal observation after 15 years of seeing this argument surface in every company I've been in) a technical need, but a combination of organizational and business requirements. This is not to say it's never technical, just that it's rare for it to be a reason.

Organizational: The teams are so huge that they prefer their own development cycle, deployment, and rollout safeguards.

Business: Partitioning of products for different SLAs, which is difficult to guarantee when you have a different team introducing some bug every other week that takes down the servers.

Re: Keep the monolith, but split the workloads

#12
post #2

Author here, thanks for sharing! This is a strategy I’ve used across many projects in several languages, from big Rails apps to the Go monolith we deploy at incident.io today. It’s just one way you can make a monolith much more robust without moving into separate services, which helps you keep the benefits of a monolithic application for longer. Hope people find it interesting.

Thank you for sharing this article! > But having spent half a decade stewarding a Ruby monolith from 20 to 200 engineers and watched its modest 10GB Postgres database grow beyond 5TB, there’s definitely a point where the pain outweighs the benefits. Woah! I for one would also love to hear your insights on scaling personnel from 20 to 200. We’re in a similar boat / anticipated growth phase (at 20 odd engineers), and w…

Glad you enjoyed the article!

The experience I reference was from my time at GoCardless, where I was a Principal SRE leading the team who were on-call for the big Rails monolith.

I’ll put the topic of “what does 10x eng team scaling feel like” on my todo list for posts, but if you’re interested there’s a bunch of articles on my personal blog from this time.

One that might be interesting is “Growing into Platform Engineering” about how the SRE function evolved:

https://blog.lawrencejones.dev/growing-into-platform-enginee...

Re: Keep the monolith, but split the workloads

#13

Or maybe modular monolith? Modulith? https://debugagent.com/is-it-time-to-go-back-to-the-monolith

Also affectionately called macroservices.

No idea if I stole it some where or came up with the idea when I was falling into slumber after losing a few days to a P0 incident, but I think it describes the N-tier architecture and what I adopt at work quite well.

Re: Keep the monolith, but split the workloads

#15
One often finds derogatory remarks about PHP, or that the popularity of the language is declining. Interestingly, the concept of PHP prevents exactly such problems, as a monolith written in PHP only ever executes the code paths that are necessary for the respective workload. The failure that the Rails app experienced in the post simply wouldn't have happened with PHP. Especially for web applications, PHP's concept of "One Request = One freshly started process" is absolutely brilliant.

So, OP is right, write more monoliths, and I would add: write them in languages that fit the stateless nature of http requests.

Re: Keep the monolith, but split the workloads

#16
> Bad code in a specific part of the codebase bringing down the whole app, as in our November incident.

This is a non-issue if you're using a Elixir/Erlang monolith given its fault tolerant nature.

The noisy neighbour issue (resource hogging) is still something you need to manage though. If you use something like Oban[1] (for background job queues and cron jobs), you can set both local and global limits. Local being a single node, and global across the cluster.

Operating in a shared cluster (vs split workload deployments) give you the benefit of being much more efficient with your hardware. I've heard many stories of massive infra savings due to moving to an Elixir/Erlang system.

1. https://github.com/sorentwo/oban

Re: Keep the monolith, but split the workloads

#17

One critique I have is that this presents a binary option of either full monolith and microservices. The truth is somewhere in between where you have dependent services large enough to warrant being their own monolith being split off. Breaking up of a monolith is almost never (anecdotal observation after 15 years of seeing this argument surface in every company I've been in) a technical need, but a combination of org…

You’re right, that’s a very fair point. I should’ve made it clear that the middle ground of moderately-sized but logically separate services is very much a good world to be in.

I’d actively encourage splitting a monolith into services when the candidate split would be:

1. A service that has a clear purpose that can be explained either separately from the monolithic whole or as a complementary piece

2. It will receive enough active development that the cost of upgrading/on-going investment is an acceptable percent of total work time

3. It can improve the organisations state of ‘ownership’ and better align the codebase with the team structure that own it

There is always a middle ground, and I should’ve done better to explain that.

Re: Keep the monolith, but split the workloads

#18

One often finds derogatory remarks about PHP, or that the popularity of the language is declining. Interestingly, the concept of PHP prevents exactly such problems, as a monolith written in PHP only ever executes the code paths that are necessary for the respective workload. The failure that the Rails app experienced in the post simply wouldn't have happened with PHP. Especially for web applications, PHP's concept of…

Small note that the app in the post was not Rails, it’s a Golang app.

You’re right in that PHP couldn’t fail like this though, as the isolation level is at a unix process, where the OS should contain all your errors.

Re: Keep the monolith, but split the workloads

#19

One often finds derogatory remarks about PHP, or that the popularity of the language is declining. Interestingly, the concept of PHP prevents exactly such problems, as a monolith written in PHP only ever executes the code paths that are necessary for the respective workload. The failure that the Rails app experienced in the post simply wouldn't have happened with PHP. Especially for web applications, PHP's concept of…

PHP is amazing, yes the syntax is a little awkward. I love working with it.

Especially with libraries/ framework that are 10+ years old Their documentation is extremely detailed.

So you have more confidence that you'll deliver a working product. Instead of reading github issues to fix an obscure error message.

Old does not always mean outdated.

Post reply on HN