Note, I'm definitely not a web developer. I've put static web pages together and fooled around with Rails & Heroku a little. So, as an outsider, I'm curious: How did you all (the web development community) make DEPLOYMENT so complicated? All these packages and environments and VMs and containers and environment variables. Shouldn't deploying something on the web be a few minutes of FTP and then go do something more p…
If you have a static website, maybe. If you have a web application with millions of users, several large and complex data stores, and lots of functionality, things get complicated fast. Most critically, it's not a direct relationship. If you double the number of pieces in your infrastructure, your problems become four times as hard if not more. You might have a database and a web app. But the database gets slow, so y…
It would be useful if there was a guide on how to scale your app wisely. > So what are the basic steps you will go through? > What are caveats and best practices per step? > What are the important choices and when do you make them?
Could you recommend a resource for answers on these questions?
I took your comment and edited it a bit:
Start: database + web app
Slow database: adding caching layer
Need to backup: starting back up all your users' data
App gets complicated, deployments break: being able to roll back quickly
You have a dozen or more servers. If one goes down everything breaks: adding redundancy to each part of the system and improve backups and recovery.
You need search features: add another kind of database to index your primary database and add more app servers.
Now when things break, it becomes really hard to tell why things are broken: add monitoring and centralized logging. Those are two more kinds of databases and web applications to manage them.
Coordinate multiple pieces of the system for any change you make: add additional software to manage that.
You'll realize your original code was terrible: you take various pieces of your infrastructure and break them into microservices in new languages.
And...