Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We decided to move 90% of our workload from the cloud to on-prem infrastructure

medium.com

Post reply on HN
Print view View original

161–170 of 218 posts

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#161

Post by lsaferite »

simonebrunozzi wrote: post #148
the_duke wrote: post #59

Earlier quoted context omitted.

In addition: long before AWS you could easily rent virtualized or dedicated servers. It took longer to provision than EC2 and all you had for storage is a fixed amount of disk, but that was absolutely sufficient for many businesses.

> In addition: long before AWS you could easily rent virtualized or dedicated servers. I disagree with this statement. Yes, you could rent, but not by the hour and based on compute power, and couldn't rent extra storage again by the hour and by the GB. Plus, you couldn't interact with these "virtual servers" through APIs. I was at AWS 2008-2014 (early days!), and I think you should consider the impact of the "on-dema…

>> In addition: long before AWS you could easily rent virtualized or dedicated servers. > I disagree with this statement.

Your disagreement here is without merit. You are talking about facets that were never even mentioned by the GP. If you wanted to list those things off as why the previous situation was suboptimal, fine, do that. But how can you disagree with a (presumably) completely factual statement?

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#162

Post by [deleted] »

manigandham wrote: post #9

> "Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS" Nonsense. There were plenty of SaaS startups. There was even a little event called the dotcom boom all about internet companies. This lack of history and experience is why new companies get into this cloud-first mess in the first place. Cloud is primarily for flexibility in iteration, dynamic scaling, or…

[deleted]

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#163

Post by bsder »

duxup wrote: post #79
SpaceMartini wrote: post #69

I work in HPC for a cloud provider, and fully endorse this move. Anonymously, of course. You can make an economic argument for or against cloud in practically every IT domain, but in HPC the case for on-prem is really compelling; none of the cloud networking/resiliency value-add is relevant to batch workflows, and costs per core-hour are only remotely comparable if you use spot - which is itself a major compromise. T…

I'm curious what people are spending / over spending all their money on in the cloud? My exposure to the actual granular costs and billing have only been limited to a small company and in that case the costs were pretty appealing compared to running everything yourself. Granted this was also a bit of a hybrid with some services local and others in the cloud. I've not had much exposure to where the deep costs start to…

> I'm curious what people are spending / over spending all their money on in the cloud?

Outbound data.

Cloud companies generally make inbound data close to free, but outbound data incredibly expensive.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#164

Post by TheCoelacanth »

bob1029 wrote: post #141
rhplus wrote: post #129

Earlier quoted context omitted.

1 million/sec is basically line speed on a 10Gbps link if each request is coming in at MTU of 1500 bytes. Sure, you might be able to push that much data through a VM on a test bench with well-behaved local clients, but you ain’t gonna be doing that rate once you add TLS, authz, logging, throttling, non-trivial serialization, non-trivial database access, A/B tests, metrics, fraud detection, recommendations, and everyt…

1500 bytes is a pretty big average payload size when you consider information theory and what actually must be communicated for this kind of business (on average). A user clicking "Watch later" on a video could theoretically be communicated in something as small as 64-bit integer for the user/session id, one for the command type, and another for the identity of the actual video. With serialization, padding, etc., you…

1500 bytes isn't even enough to send a list of video titles and thumbnail images for a single page of videos.

If you imagine that the client already has a full database of all the available videos and metadata about them, then you could get by with tiny amounts of data, but that's not even close to the actual circumstances that Netflix operates under.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#165

Post by Cpoll »

charcircuit wrote: post #51
chrxr wrote: post #24

Earlier quoted context omitted.

Worth noting that using a gaming card for workloads like this would likely void the warranty.

No it doesn't

Isn't it right there in the fine print?

> Warranted Product is intended for consumer end user purposes only, and is not intended for datacenter use and/or GPU cluster commercial deployments ("Enterprise Use"). Any use of Warranted Product for Enterprise Use shall void this warranty. https://www.nvidia.com/en-us/support/warranty/

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#166

Post by dhzhzjsbevs »

BryanRamirez wrote: post #137
dhzhzjsbevs wrote: post #105

Earlier quoted context omitted.

The cost isn't gone. It's just included. Fact is, at scale, it's still cheaper to do it yourself. You just need to reach the scale it's worth paying people to do the managing.

I agree, with the caveat that at scale it's cheaper to do it well yourself. If that weren't true, then Amazon would not be making money on EC2. If you don't have sysadmins and network admins with the right experience, you can easily find yourself in a bad spot with single points of failure, servers that can't be easily replaced, oversubscribed PDUs, misconfigured switches/routers... and any number of other problems t…

This is very true. You also probably need the scale having multiple install locations makes sense.

We went dedicated early and while it didn't make sense at the time we now run at way lower cost than the competition could dream of.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#167

Post by [deleted] »

phonon wrote: post #114
post-it wrote: post #104

Earlier quoted context omitted.

The biggest thing for me is no surprise bills. Sure, it's pretty unlikely that a t2.small instance and its associated storage and network subscriptions are going to produce a $1000 bill one day, but there's literally no way to set a hard cap on billing so clearly Amazon thinks it might be possible.

Use https://aws.amazon.com/lightsail/pricing/ then...

[deleted]

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#168

Post by post-it »

whiplash451 wrote: post #132
the_duke wrote: post #59

Earlier quoted context omitted.

In addition: long before AWS you could easily rent virtualized or dedicated servers. It took longer to provision than EC2 and all you had for storage is a fixed amount of disk, but that was absolutely sufficient for many businesses.

OVH has a great dedicated server offering. Definitely a great bang for the buck compared to AWS (if you can handle the downsides of course: security, backup, setup, handled by yourself).

> OVH

Emphasis on backup.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#169

Post by earleybird »

smm11 wrote: post #122
briffle wrote: post #96

Earlier quoted context omitted.

So nobody got fired for buying IBM?

I can hang 32 terminals off just one PC. You're still blowing your budget on standalones.

Dang it all, we had 20+ teletypes hanging off a 32Kb PDP-8 (with DECTape for random access storage), "and we liked it!" :-)

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#170

Post by nijave »

>Adding storage, CPU time, and many other costs makes understanding and verifying the cost structure a task suitable for certified experts

Is that easier on-prem? I was under the impression it was even more difficult--especially with shared tenancy (how much incremental cost does App A auth add to our Active Directory deployment?)

You're going to need to know server power utilization under load to calculate power/cooling costs and probably some additional data on network utilization to figure out incremental costs for that

Maybe if it's a colo or managed data center that gets rolled up for you, but if you're managing yourself, you still have to figure it out

Blog post also doesn't mention cost of downtime (maybe not an issue for them) or a metrics solution (you usually get basic machine and service metrics for free on the big cloud providers)

Post reply on HN