We decided to move 90% of our workload from the cloud to on-prem infrastructure
1–10 of 218 posts
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#2> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute.
Most don't. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per second is totally reasonable too if you are careful with a few things...
I genuinely believe you could put the literal public Netflix biz experience on a single VM. Account management, billing, preferences, view history, etc. The only pieces that need cloud scale are ddos mitigation, 4k video streams and media-dense static web content. Most businesses do not have strong need for these things.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#3 After spending $100k in a year.. After doing the math, We decided to purchase three workstations at a total cost of $17k.
One is a GPU-based workstation with two RTX 3090s and an Intel i9–12900 CPU, and another two workstations with 16 cores AMD Ryzen 5950X CPUs.
It took us a few FTE days to set those up to our satisfaction with slurm, NFS, backups, and several other services.
We noticed that our RTXs, although considered gaming cards, are comparable (if not better) in performance to Tesla V100, which some cloud providers rent at the staggering price of $3.06 an hour.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#4I hate the fact that hiring now basically requires cloud experience specific to vendors. This is basically going to force people into one of the three major cloud platforms.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#5> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Most don't. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per second is totally reasonable too if you are careful with a few things... I genuinely believe you could put the literal public Netflix biz experience on a single VM.…
Correctish in a sense: https://openconnect.netflix.com/en/appliances/#the-hardware
They throw one of these boxes at an ISP and interconnect. 4-6 year no touch reliability, couple hundred TB storage.
Modern hardware is quite something. They will saturate 2x100GE. That's in the thousands of concurrent streams per box.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#6This is a good explanation of cloud issues for a company with resources and consistent workload...
IaaS is not really competitive in this space, I don't think... If you have access to system admins, and have a consistent work load, you could avoid the cloud, trading the cloud premium for more employees and skills in your team. This is fine, if that is what you're team needs.
The cloud is not a silver bullet that solves all companies infrastructure... But they have a very profitable space, especially in small businesses, or businesses that benefit from multiple data centers. AWS simply made it easy to scale up and down, as well as scale around the world... if you don't need to dynamically scale or easy access to multiple data centers, the cloud begins to lose it's best (cost-effective) competitive edge to self hosting... Though at the micro scale, the cloud can do dead simple basics for free -- or near enough (e.g. static websites), which is fun for personal projects
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#7After spending $100k in a year.. After doing the math, We decided to purchase three workstations at a total cost of $17k. One is a GPU-based workstation with two RTX 3090s and an Intel i9–12900 CPU, and another two workstations with 16 cores AMD Ryzen 5950X CPUs. It took us a few FTE days to set those up to our satisfaction with slurm, NFS, backups, and several other services. We noticed that our RTXs, although consi…
> They did the math.
And bought 3 workstations instead of 3 servers.
IMHO the headline of "Why Enzymit Decided to Build its Own On-Prem HPC Infrastructure" is a bit... stretched.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#8Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS (Infrastructure as a Service) companies. There were simply too many hurdles, and the expenses were too high — you’d have to purchase dedicated servers and a high bandwidth connection to handle a load of incoming visitors to your website, hire engineers to build a scalable system, and if you planned to go international, you would have to purchase servers in other geographical locations.
The first internet boom was exactly what the author describes. I worked for an internet e-commerce start-up in 1999, and guess what? We had dedicated hosting bandwidth coming into our office, and the hardware and software in place to serve our application. Of course, one of the founders was a sysadmin, but I had friends working for similar start-ups, and they leveraged one of the many co-location providers in the city to manage all the details. Yet their employer still owned actual hardware that they could touch when necessary.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#9> "Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS"
Nonsense. There were plenty of SaaS startups. There was even a little event called the dotcom boom all about internet companies. This lack of history and experience is why new companies get into this cloud-first mess in the first place.
Cloud is primarily for flexibility in iteration, dynamic scaling, or complex configurations that would be otherwise hard to do. If you have steady-state load like this then a few servers in your colo is pretty simple and far cheaper.
Companies also vastly overestimate their scale when their entire business could probably fit on a single commodity server.
Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure
#10After spending $100k in a year.. After doing the math, We decided to purchase three workstations at a total cost of $17k. One is a GPU-based workstation with two RTX 3090s and an Intel i9–12900 CPU, and another two workstations with 16 cores AMD Ryzen 5950X CPUs. It took us a few FTE days to set those up to our satisfaction with slurm, NFS, backups, and several other services. We noticed that our RTXs, although consi…
They also had a static workload, with no changing requirements, or really any need for scalability.
They also had FTEs with experience to configure the systems (which for a lot of us technical people is a no-brainer, but if you had data scientists with no hardware experience, that might be different)
This worked for them, and I'm happy for them.. the cloud does not solve all problems... It's simply one hardware strategy you can pick.. it's important to review the options!