Does anyone have a similar guide for scaling AI systems? I am afraid this architecture cannot scale to 10M+ users serving LLMs for example.
How to Scale a System from 0 to 10M+ Users
61–70 of 82 posts
Re: How to Scale a System from 0 to 10M+ Users
#62Earlier quoted context omitted.
> And how do you predict with certainty your "highest expected load"? And if you're in a space like ecommerce, where you have 1 week out of the year with x10 or x50 the load, I doubt it would actually be cheaper than using autoscaling. If you know when that week is, where's the problem in spinning up extra capacity just for that period , a week in advance? Retailers, whether online or not, know with a very high confi…
But you have to buy those servers, set them up, use them for that week, and then they're left idle for the whole of the year.
> But you have to buy those servers, set them up, use them for that week, and then they're left idle for the whole of the year.
I didn't say you have to buy them. I said you can spin up extra capacity.
Re: How to Scale a System from 0 to 10M+ Users
#63Earlier quoted context omitted.
> their original idea was to play with shiny tech instead This is a behavior I would say is very hard to manage out of people and should be screened for aggressively in interviews.
This behavior is self-inflicted by a decade of low pay and lack of significant raises to reward seniority. The most effective way to increase income for a developer is to join a place, rack up as many buzzwords as possible and leave after 2-3 years, using those buzzwords to secure a higher-paying role somewhere else. Rinse and repeat until you get a management position where you can use politics to increase your inco…
The time up to and including Covid saw massive developer salary increases. They've dropped (and lots have been laid off) post-Covid, but the last ten years cannot be described as stagnation.
Re: How to Scale a System from 0 to 10M+ Users
#64[flagged]
My god even the comments here are AI slop.
Re: How to Scale a System from 0 to 10M+ Users
#65Can someone make a version of this that is not AI generated and ACTUALLY correct and pragmatic about the different stages with suggestions about the different scenarios that would lead to the different decisions?
Re: How to Scale a System from 0 to 10M+ Users
#66Can someone make a version of this that is not AI generated and ACTUALLY correct and pragmatic about the different stages with suggestions about the different scenarios that would lead to the different decisions?
This website itself is running on a single server and handles millions of requests per day. Raw number of users is simply not what decides the architecture, not even a little bit.
Re: How to Scale a System from 0 to 10M+ Users
#67Earlier quoted context omitted.
As is often stated, microservices is a solution for scaling an engineering org to 100s of developers, not for scaling a product to millions of users.
You can get the separation benefits of microservices in a compiled language with modules that only communicate over well-defined interfaces, constraining each team within their own module without having to introduce a network call between each operation.
Python dev is cheaper and faster though. People arent gonna kill velocity by making their backend in c++ so the devs can have seperation of concerns, something that can, and should, be self enforced with discipline
Re: How to Scale a System from 0 to 10M+ Users
#68Earlier quoted context omitted.
> their original idea was to play with shiny tech instead This is a behavior I would say is very hard to manage out of people and should be screened for aggressively in interviews.
This behavior is self-inflicted by a decade of low pay and lack of significant raises to reward seniority. The most effective way to increase income for a developer is to join a place, rack up as many buzzwords as possible and leave after 2-3 years, using those buzzwords to secure a higher-paying role somewhere else. Rinse and repeat until you get a management position where you can use politics to increase your inco…
I wouldnt describe the 2010s as low pay for devs
Re: How to Scale a System from 0 to 10M+ Users
#69Earlier quoted context omitted.
I once worked with a startup that had 3 total engineers, and their architecture diagram called for 8 micro services. We tore that architecture diagram up pronto
Out of interest how do you get the authority to make those decisions and have the existing developers continue working productively after this? To me it seems like microservices (or cloud, or whatever) is often overused for career/buzzword reasons. The engineers pushing for it aren't asking for your advice, they want to build an engineering playground - denying them the opportunity is unlikely to suddenly make them p…
> Out of interest how do you... have the existing developers continue working productively after this?
I don't know if you can in general - by happy coincidence the engineer who had made that plan found his way to the door shortly thereafter
Re: How to Scale a System from 0 to 10M+ Users
#70Earlier quoted context omitted.
As is often stated, microservices is a solution for scaling an engineering org to 100s of developers, not for scaling a product to millions of users.
Unfortunately that message was way way behind the bombast of "microservices everywhere now" that preceded it for years, to the detriment of many small orgs. I've seen engineering orgs of 10-50 launch headlong into microservices to poor results. No exaggeration to say many places ended up with more repos & services than developers to manage them.
The worst I ever saw was an engineer misunderstood microservices, and made a service per endpoint.
He started complaining to management that 50 CI/CD setups was his limit he could support.
He was absolutely amazed when I showed him he could combine endpoints into a larger logical service. 50 services became three, and it’s still three a few years later now.