Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We decided to move 90% of our workload from the cloud to on-prem infrastructure

medium.com

Post reply on HN
Print view View original

81–90 of 218 posts

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#81

Post by dagw »

SpaceMartini wrote: post #69

I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really compelling; none of the cloud networking/resiliency value-add is relevant to batch workflows, and costs per core-hour are only remotely comparable if you use spot - which is itself a major compromise. T…

The only real advantage cloud has for science is object storage

As someone who has done a fair bit of HPC I consider the real advantage to be temporary scalability. If my 'normal' compute notes have 128 GB of RAM and all of a sudden I have job that need 300 GB or RAM, with cloud I can just change a line in a config file and run that calculation on a machine with 300 GB of RAM. Or if I have a job that will optimally run on 100s of 1-core machines with only 4 GB of RAM I can set up a cluster of such machines with in minutes.

That being said I 100% agree that if you have a normal baseline workload that should absolutely be done on in house hardware.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#82

Post by fxtentacle »

v-yadli wrote: post #66
GordonS wrote: post #60

Earlier quoted context omitted.

> I was positively surprised that 10G fiber networking is now down to $70 for a PCIe card + 20m cable kit. Wow, that is surprising, maybe it's time I started upgrading my LAN...

Actually if you have cat5 or cat6 base-T setup, you can get second hand dual port 10gbe pcie cards for 30 dollars each(!) 10GbE baseT switches are more expensive than fiber equivalent though.

In my case, cat6 cables and the added price of a 10G RJ-45 switch would have been more expensive than a 10G SPF+ switch and some twinax & fiber cables. Amazon has finished SPF-to-SPF assemblies for €10=$12. And for TP-Link, RJ-45 is like 1.5x the price of SPF+ equipment. Also, RJ-45 has a fixed minimum latency, due to it needing to support backwards-compatibility with 1G 100M etc. That means you need much larger send/transmit buffers to saturate the link as ping goes up. Especially if you need multiple hops, fiber is just faster. In my tests, 0.2ms vs 3ms roundtrip time for 10G SPF vs. 10G RJ-45.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#83

Post by [deleted] »

tiew9Vii wrote: post #74
maccard wrote: post #43

Earlier quoted context omitted.

> The most expensive part is the electricity. No, the most expensive part is the persons time for managing it. I can rent a monstrous Dedicated Server for $400/month from OVH, but even with a UK salary, if I have to spend more than 1 day a month on it in any shape or form (and that includes the initial setup), it's cheaper to use "the cloud" or some form of a managed service.

I hear that said a lot. I don’t think it stands. A ec2 instance or other vps requires the exact same maintenance as a bare metal server. They are essentially the same except one is virtualised and the other isn’t. The cloud actually requires more investment for large organisations. Previously you might of only had a handful of sysadmins but now you have a large dedicated platform team doing devops type work building…

[deleted]

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#84

Post by karmakaze »

bob1029 wrote: post #2

> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Most don't. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per second is totally reasonable too if you are careful with a few things... I genuinely believe you could put the literal public Netflix biz experience on a single VM.…

Here's more of that context that paints a better picture.

> First, it’s important to note that Enzymit’s use of cloud computing mainly entailed computationally intensive calculations for protein design. We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Our primary use case is running CPU and GPU heavy analyses, and for that use case, we have found the IaaS/public cloud solution to be far from cost-effective in the long term.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#85

Post by sofixa »

fxtentacle wrote: post #61
GordonS wrote: post #60

Earlier quoted context omitted.

> I was positively surprised that 10G fiber networking is now down to $70 for a PCIe card + 20m cable kit. Wow, that is surprising, maybe it's time I started upgrading my LAN...

I use Intel 82599ES SFP+ and a TL-SX3008F router. But let me warn you: Things are affordable, but NOT consumer-friendly. I needed to study the 500 page PDF manual and do basic link configuration through telnet via USB before I could connect to the router via Ethernet and use its web-GUI to finish the setup.

How's the routing performance on that switch? I'm really struggling finding a device to fit my needs ( routing, 6-8 ports, Ethernet, at least two 2.5/5/10G).

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#86

Post by wongarsu »

manigandham wrote: post #62
the_duke wrote: post #59

Earlier quoted context omitted.

In addition: long before AWS you could easily rent virtualized or dedicated servers. It took longer to provision than EC2 and all you had for storage is a fixed amount of disk, but that was absolutely sufficient for many businesses.

Definitely more process involved, but the requirements were also simpler compared to sprawling modern architectures. I think the overall effort is similar, but was slower due to the speed of communications and paperwork at that time. Getting hard assets is a core part of business in basically every industry, so I found it strange to claim that it was some major obstacle just because it happened to be servers instead…

I can only speak for the last 15 years or so, but any time I rented virtual or dedicated servers the experience was basically the same as ordering a book from Amazon: you create an account, select what you want and how you want to pay, the next day you have an email with IP and credentials.

Of course since then things have improved and you can now expect your sever to be provisioned within minutes, along with a nice dashboard to manage it. It's a bit more involved if you want to build your own server and put it in colocation somewhere, mostly because that involves being physically present.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#87

Post by stephbu »

tiew9Vii wrote: post #74
maccard wrote: post #43

Earlier quoted context omitted.

> The most expensive part is the electricity. No, the most expensive part is the persons time for managing it. I can rent a monstrous Dedicated Server for $400/month from OVH, but even with a UK salary, if I have to spend more than 1 day a month on it in any shape or form (and that includes the initial setup), it's cheaper to use "the cloud" or some form of a managed service.

I hear that said a lot. I don’t think it stands. A ec2 instance or other vps requires the exact same maintenance as a bare metal server. They are essentially the same except one is virtualised and the other isn’t. The cloud actually requires more investment for large organisations. Previously you might of only had a handful of sysadmins but now you have a large dedicated platform team doing devops type work building…

I broadly agree with the statement that owning an OS in general is toil that applies any of these IaaS/VPS/BM scenarios.

Owning a BM server different toil - server parts fail, the network it attaches to needs control and it fails too, firmware needs updating more regularly than ever, DC space needs managing over time etc. For a small number of machines maybe this is NBD. For thousands of machines this is just grunt work which while automated, still needs change management and control - rebooting the whole fleet in the middle of the day definitely opens doors in your career.

Doing everything you did in the DC in the Cloud is absolutely the worst way to adopt Cloud. Owning an OS is a non-goal, you’ve gotta climb to a higher abstraction - workloads, and quit caring about machines. This is where most companies fail.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#88

Post by philliphaydon »

tiew9Vii wrote: post #74
maccard wrote: post #43

Earlier quoted context omitted.

> The most expensive part is the electricity. No, the most expensive part is the persons time for managing it. I can rent a monstrous Dedicated Server for $400/month from OVH, but even with a UK salary, if I have to spend more than 1 day a month on it in any shape or form (and that includes the initial setup), it's cheaper to use "the cloud" or some form of a managed service.

I hear that said a lot. I don’t think it stands. A ec2 instance or other vps requires the exact same maintenance as a bare metal server. They are essentially the same except one is virtualised and the other isn’t. The cloud actually requires more investment for large organisations. Previously you might of only had a handful of sysadmins but now you have a large dedicated platform team doing devops type work building…

> A ec2 instance or other vps requires the exact same maintenance as a bare metal server. They are essentially the same except one is virtualised and the other isn’t.

Ani. If the hardware you’re running on is dying. In ec2 you stop it and start it. It’s on new hardware. If you run bare metal you’re screwed.

Disk is dying ? Don’t matter cos your data exists multiple times over in AWS elastic storage. With bare metal you got to shut down and replace.

The cost of managing hardware is gone with ec2.

Software on the other hand yes that is the same amount of effort.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#89

Post by maccard »

tiew9Vii wrote: post #74
maccard wrote: post #43

Earlier quoted context omitted.

> The most expensive part is the electricity. No, the most expensive part is the persons time for managing it. I can rent a monstrous Dedicated Server for $400/month from OVH, but even with a UK salary, if I have to spend more than 1 day a month on it in any shape or form (and that includes the initial setup), it's cheaper to use "the cloud" or some form of a managed service.

I hear that said a lot. I don’t think it stands. A ec2 instance or other vps requires the exact same maintenance as a bare metal server. They are essentially the same except one is virtualised and the other isn’t. The cloud actually requires more investment for large organisations. Previously you might of only had a handful of sysadmins but now you have a large dedicated platform team doing devops type work building…

> I think the more PaaS like services such as Heroku, Lambda, Fargate, Google Cloud Run do better realise the less maintenance story but not cloud generally.

Completely agree here. Running an EC2 Spot instance + ebs volumes for 730 hours a month is a total waste of money, but running RDS behind fargate and ECS with an ALB is likely to save you time and money by month 2.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#90

Post by _hcuq »

the_duke wrote: post #59
manigandham wrote: post #9

> "Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS" Nonsense. There were plenty of SaaS startups. There was even a little event called the dotcom boom all about internet companies. This lack of history and experience is why new companies get into this cloud-first mess in the first place. Cloud is primarily for flexibility in iteration, dynamic scaling, or…

In addition: long before AWS you could easily rent virtualized or dedicated servers. It took longer to provision than EC2 and all you had for storage is a fixed amount of disk, but that was absolutely sufficient for many businesses.

I still use them. Easier than fiddling with AWS.
Post reply on HN