Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We decided to move 90% of our workload from the cloud to on-prem infrastructure

medium.com

Post reply on HN
Print view View original

121–130 of 218 posts

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#121

Post by lmeyerov »

SpaceMartini wrote: post #69

I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really compelling; none of the cloud networking/resiliency value-add is relevant to batch workflows, and costs per core-hour are only remotely comparable if you use spot - which is itself a major compromise. T…

Yep it is quite an arbitrage going on. Unless startup credits at the highest tier ($100K), way too expensive for most startups.

For a lot of our customers, cloud is impossible now for HPC: shortages are so bad that you have to know someone high up at top cloud providers to get access to right-sized GPUs. (T4? Forget it -- one of our tickets is open since ~Christmas.)

We have gone hybrid, and for growing compute, going multi-cloud, with main stuff on top 3 clouds (CPU, light minimal GPU...), and GPUs elasticity on other ones. And for a lot... Yep, just buy GPUs for local dev.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#122

Post by smm11 »

briffle wrote: post #96
SpaceMartini wrote: post #73

Earlier quoted context omitted.

As an addendum to this: if you absolutely must use cloud, stick with AWS. Using Azure is (IMO) a fucking miserable experience and their only advantage (InfiniBand) is better served by buying your own hardware. GCP and OCI might be fine if you are getting a lot of credits, but the skills will not be useful down the line - while AWS is expensive, you will at least learn a bunch of in-demand operational skills.

So nobody got fired for buying IBM?

I can hang 32 terminals off just one PC. You're still blowing your budget on standalones.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#123

Post by chromaton »

emteycz wrote: post #113
wongarsu wrote: post #86

Earlier quoted context omitted.

I can only speak for the last 15 years or so, but any time I rented virtual or dedicated servers the experience was basically the same as ordering a book from Amazon: you create an account, select what you want and how you want to pay, the next day you have an email with IP and credentials. Of course since then things have improved and you can now expect your sever to be provisioned within minutes, along with a nice…

Well speaking about the last 15 years, so 2007 to today, for me it was the same as modern clouds - register, add a credit card, order a server, few minutes later the root password arrives in e-mail. And oldschool LAMP webhosting is basically serverless cloud. Edit: Actually I remember using actual cloud provider back in 2008 - it was called Virtualmaster, one of the first cloud providers in central EU. They offered a…

Linode was founded in 2003.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#124

Post by vmception »

the_duke wrote: post #59
manigandham wrote: post #9

> "Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS" Nonsense. There were plenty of SaaS startups. There was even a little event called the dotcom boom all about internet companies. This lack of history and experience is why new companies get into this cloud-first mess in the first place. Cloud is primarily for flexibility in iteration, dynamic scaling, or…

In addition: long before AWS you could easily rent virtualized or dedicated servers. It took longer to provision than EC2 and all you had for storage is a fixed amount of disk, but that was absolutely sufficient for many businesses.

> but that was absolutely sufficient for many businesses.

yes, but I am glad about how many job titles are obsoleted for many businesses due to how compute instances are managed.

much smaller organizations used to need a full blown database administrator or two, and other personnel dedicated to keeping the server up. or you were doing it all yourself and spending your time on that.

much higher barrier of entry than today where you have an untold number of computers spun up for you in an instant and a bunch of cached versions on yet more computers in the CDN and another process keeping those caches updated, with you just thinking its one single instance used because you're on the hobby plan.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#125

Post by BryanRamirez »

briffle wrote: post #94
bsenftner wrote: post #22

Earlier quoted context omitted.

Yes, they are cheap. Running one's own server is also easy peasy; far too many think it is difficult, it is not. The most expensive part is the electricity.

Been a sysadmin for many years. Your right, for a few computers. Once you start getting into more than a quarter rack, you also need to start worrying about cooling (which also is lots of electricity) and usually things like ensuring the electricity stays on. (UPS, generator, etc). Don't forget to monitor and service all this stuff regularly. Once your past a few racks of equipment, you have generator tests and servi…

(I created an account just to upvote and comment on this)

Absolutely, nevermind managing OS upgrades and needing to use configuration management to mitigate against drift. It took a lot of effort to make sure that each server was not a special snowflake that could not be reliably reproduced.

Also, dealing with vendor warranties, and being on hold with HP (or whoever), then assuring them you're running the latest firmware.. please for the love of god just replace the failed memory/disk/cpu.

I found the sheer physicality of computing infrastructure to be a source of exhaustion and burnout. It's a big part of the reason why I'm a software developer now :)

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#126

Post by marcosdumay »

stephbu wrote: post #63
xuki wrote: post #14

Earlier quoted context omitted.

I'd say don't even consider colo unless you have a specific use case. Rented dedicated servers are cheap, let someone else take care of the hardware.

To add to this statement - don’t make any investment unless you have a solid plan to fully utilize it for the full length of the purchase terms. This applies as much to AWS @ minute terms, as it does to a rental @ daily terms, as it does to co-lo capital investment @ yearly terms.

> unless you have a solid plan to fully utilize it for the full length of the purchase

Hum... I'd say it's much more reasonable to look at the ROI. Making investments to supply peak demand or to hedge against rare risks is perfectly ok.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#127

Post by SpaceMartini »

dagw wrote: post #81
SpaceMartini wrote: post #69

I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really compelling; none of the cloud networking/resiliency value-add is relevant to batch workflows, and costs per core-hour are only remotely comparable if you use spot - which is itself a major compromise. T…

The only real advantage cloud has for science is object storage As someone who has done a fair bit of HPC I consider the real advantage to be temporary scalability. If my 'normal' compute notes have 128 GB of RAM and all of a sudden I have job that need 300 GB or RAM, with cloud I can just change a line in a config file and run that calculation on a machine with 300 GB of RAM. Or if I have a job that will optimally r…

Those are fair points - I have seen truly spiky workloads like that very occasionally, but more often those spikes are a precursor to more sustained usage in a similar manner and so would quickly warrant hardware purchases.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#128

Post by SpaceMartini »

Tsarbomb wrote: post #78
SpaceMartini wrote: post #69

I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really compelling; none of the cloud networking/resiliency value-add is relevant to batch workflows, and costs per core-hour are only remotely comparable if you use spot - which is itself a major compromise. T…

If you can setup HPC at scale you can setup Ceph for object storage. It will save you so much money in the long run.

I have heard too many horror stories about Ceph (and OpenStack) to be confident about that. I certainly don't think I can truly beat S3 on cost or performance at the terabyte scale for household data - and while larger scale would give on-prem savings there are also higher expectations (in terms of availability and performance) of a multi-perabyte storage array.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#129

Post by rhplus »

bob1029 wrote: post #2

> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Most don't. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per second is totally reasonable too if you are careful with a few things... I genuinely believe you could put the literal public Netflix biz experience on a single VM.…

1 million/sec is basically line speed on a 10Gbps link if each request is coming in at MTU of 1500 bytes. Sure, you might be able to push that much data through a VM on a test bench with well-behaved local clients, but you ain’t gonna be doing that rate once you add TLS, authz, logging, throttling, non-trivial serialization, non-trivial database access, A/B tests, metrics, fraud detection, recommendations, and everything else that makes an API like Netflix tick. Whether you do that all on one VM or split across service roles, you’re gonna be much more realistically in the range of 1000 rps per CPU core.
Post reply on HN