Keep the monolith, but split the workloads
incident.io
Keep the monolith, but split the workloads
1–10 of 161 posts
Re: Keep the monolith, but split the workloads
#2Author here, thanks for sharing!
This is a strategy I’ve used across many projects in several languages, from big Rails apps to the Go monolith we deploy at incident.io today.
It’s just one way you can make a monolith much more robust without moving into separate services, which helps you keep the benefits of a monolithic application for longer.
Hope people find it interesting.
Re: Keep the monolith, but split the workloads
#3This is standard practice on monolithic apps. It is effectively an N-tier architecture. On Rails it is very expected that you would deploy the same codebase to two tiers (web and worker) and simply run rails s in one and sidekiq in the other.
Re: Keep the monolith, but split the workloads
#4This is standard practice on monolithic apps. It is effectively an N-tier architecture. On Rails it is very expected that you would deploy the same codebase to two tiers (web and worker) and simply run rails s in one and sidekiq in the other.
We're doing this with our monolithic Node.js / Express application as well, but I wasn't aware that it's a common pattern. For us having a monolithic app greatly simplifies development, deployment, updating dependencies, shared code, etc. Just having a single set of third-party dependencies to update makes things so much easier.
Re: Keep the monolith, but split the workloads
#5I came to the same approach in a past job.
One minor problem was that some functionality of that monolith was based on keeping big chunks of DB in memory. So on separated workloads, the instances dedicated to serve other functions, were keeping unnecessary data in memory, putting unneeded load on DB during startup. This could be solved by introducing some startup parameters. But for beginning this problem can be ignored.
Re: Keep the monolith, but split the workloads
#6Or maybe modular monolith? Modulith?
https://debugagent.com/is-it-time-to-go-back-to-the-monolith
Re: Keep the monolith, but split the workloads
#7This is standard practice on monolithic apps. It is effectively an N-tier architecture. On Rails it is very expected that you would deploy the same codebase to two tiers (web and worker) and simply run rails s in one and sidekiq in the other.
Ditto in Django + Celery
Re: Keep the monolith, but split the workloads
#8This is standard practice on monolithic apps. It is effectively an N-tier architecture. On Rails it is very expected that you would deploy the same codebase to two tiers (web and worker) and simply run rails s in one and sidekiq in the other.
We're doing this with our monolithic Node.js / Express application as well, but I wasn't aware that it's a common pattern. For us having a monolithic app greatly simplifies development, deployment, updating dependencies, shared code, etc. Just having a single set of third-party dependencies to update makes things so much easier.
It’s ‘standard practice’ for many frameworks like Rails which encourage the pattern via a separate deployment of the async worker tier.
But outside of these frameworks, you won’t often see it. And especially in languages like Go which tend to avoid opinionated frameworks it’s common not to see a split like this, even when it might be really beneficial.
As with most programming concepts, standard practice varies a lot depending on the tools you use. It’s why I thought this post was worth writing, in that there are many people who haven’t run Rails/etc frameworks before and may not have considered this approach!
Re: Keep the monolith, but split the workloads
#9Author here, thanks for sharing! This is a strategy I’ve used across many projects in several languages, from big Rails apps to the Go monolith we deploy at incident.io today. It’s just one way you can make a monolith much more robust without moving into separate services, which helps you keep the benefits of a monolithic application for longer. Hope people find it interesting.
Thank you for sharing this article!
> But having spent half a decade stewarding a Ruby monolith from 20 to 200 engineers and watched its modest 10GB Postgres database grow beyond 5TB, there’s definitely a point where the pain outweighs the benefits.
Woah!
I for one would also love to hear your insights on scaling personnel from 20 to 200.
We’re in a similar boat / anticipated growth phase (at 20 odd engineers), and whilst there’s a lot of content on this topic, I’d appreciate your practical “from the coal face” take (much like what you’ve done with this monolith article).
Perhaps a follow up article? ^_^
Re: Keep the monolith, but split the workloads
#10Or maybe modular monolith? Modulith? https://debugagent.com/is-it-time-to-go-back-to-the-monolith
Yep, that article is about very similar concepts but grounded in Spring as the framework.
I like what they do around package imports and it looks a lot like what we do at incident.io, with some rules about which packages can import what.
For people in the Ruby world who want a similar solution, Shopify provide an open-source framework called packwerk that is designed just for this: