Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We decided to move 90% of our workload from the cloud to on-prem infrastructure

medium.com

Post reply on HN
Print view View original

31–40 of 218 posts

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#31

Post by lillecarl »

lillecarl wrote: post #28
pzduniak wrote: post #11

Earlier quoted context omitted.

Eh, it depends on the features that you're using. As long as you stick to the basic stack of Terraform + Kubernetes / IaaS with cloudinit + networking + S3-compatible storage API, you can quite easily jump between clouds. Sure, the logic that sets them up is different, but the concepts are generally roughly the same. Even if I end up choosing a managed service, I always implement a second OSS backend that gets regula…

This is what I've come to realise too, as long as you can and do stick your workloads in Kubernetes it doesn't really matter what he logo says. EKS, AKE, GKE, LKE, DOKS, OKD, Rancher... Whatever they're all compatible with what you want to do. There are definitely upsides to the cloud, but Kubernetes is the common denominator everywhere. Wanna run GPU workloads on-prem? Buy some servers and do so. The only hairy thin…

GCE https://kubernetes.io/docs/concepts/cluster-administration/c...

GKE https://cloud.google.com/container-engine/docs/cluster-autos...

AWS https://github.com/kubernetes/autoscaler/blob/master/cluster...

Azure https://github.com/kubernetes/autoscaler/blob/master/cluster...

Alibaba Cloud https://github.com/kubernetes/autoscaler/blob/master/cluster...

Brightbox https://github.com/kubernetes/autoscaler/blob/master/cluster...

OpenStack Magnum https://github.com/kubernetes/autoscaler/blob/master/cluster...

DigitalOcean https://github.com/kubernetes/autoscaler/blob/master/cluster...

CloudStack https://github.com/kubernetes/autoscaler/blob/master/cluster...

Exoscale https://github.com/kubernetes/autoscaler/blob/master/cluster...

Equinix Metal https://github.com/kubernetes/autoscaler/blob/master/cluster...

OVHcloud https://github.com/kubernetes/autoscaler/blob/master/cluster...

Linode https://github.com/kubernetes/autoscaler/blob/master/cluster...

OCI https://github.com/kubernetes/autoscaler/blob/master/cluster...

Hetzner https://github.com/kubernetes/autoscaler/blob/master/cluster...

Cluster API https://github.com/kubernetes/autoscaler/blob/master/cluster...

Vultr https://github.com/kubernetes/autoscaler/blob/master/cluster...

TencentCloud https://github.com/kubernetes/autoscaler/blob/master/cluster...

These are all cloud providers that invested into their own managed Kubernetes, I'm certain all of them aren't as sleek as the big three, but it shows that there's definitely momentum behind sticking your workloads into Kubernetes.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#32

Post by anonymousDan »

lvl102 wrote: post #4

I hate the fact that hiring now basically requires cloud experience specific to vendors. This is basically going to force people into one of the three major cloud platforms.

For someone with a good general understanding of systems (distributed systems, operating systems etc), but perhaps lacking in hands on experience of deploying in a specific cloud, what would you say the key things are to learn? For me they almost seem like trivial things you could pick up in no time on the job.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#33

Post by piker »

zmmmmm wrote: post #29

Hope he's prepared for a letter from nVidia's lawyers for breaking their license agreement for the RTX3090's.

Is all commercial use of the consumer grade nVidia products prohibited under that license? I thought (and this seems to agree: https://www.nvidia.com/en-gb/drivers/geforce-license/) that it was just use in a datacenter that was prohibited.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#34

Post by mathattack »

tonyedgecombe wrote: post #18
lioeters wrote: post #13

Just want to highlight how futuristic the author's title is: "Computational Biologist, Head of Protein Design @ Enzymit". Why they moved to on-prem: lower and more predictable cost. At a public cloud provider, they lost thousands of dollars (of free credit they had) through "architectural blunders". And the running cost of GPU, CPU, storage, and data transfer summed up to $10K a month - at which point they figured th…

>"architectural blunders" I wonder how much of this is by design. It seems it is in Amazon's interest to keep their systems and pricing as opaque as possible.

At MSFT the Azure solutions architects are part of the sales organization. Their commissions are tied to usage (revenue) which tells you everything about their skill sets. And their lack of standard clear pricing means you have to estimate everything yourself.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#35

Post by Nextgrid »

ldoughty wrote: post #6

This is a good explanation of cloud issues for a company with resources and consistent workload... IaaS is not really competitive in this space, I don't think... If you have access to system admins, and have a consistent work load, you could avoid the cloud, trading the cloud premium for more employees and skills in your team. This is fine, if that is what you're team needs. The cloud is not a silver bullet that solv…

> If you have access to system admins

Clouds also require sysadmins, they're just called "DevOps engineers" now. Those YAML & Terraform files aren't going to write themselves.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#36

Post by vidarh »

manigandham wrote: post #9

> "Starting a web-based or SaaS (Software as a Service) business was virtually unheard of before the age of IaaS" Nonsense. There were plenty of SaaS startups. There was even a little event called the dotcom boom all about internet companies. This lack of history and experience is why new companies get into this cloud-first mess in the first place. Cloud is primarily for flexibility in iteration, dynamic scaling, or…

I agree fully with this with one caveat: if you're in an location then managed servers can be cheaper than your own colo. E.g. I'm in London, and it's hard to beat Hetzner with colos near enough to me to be practical.

Still sometimes reasons to use colos, but I think it's important people consider that the choice isn't just cloud or your own equipment on prem or in a colo - managed servers can get you most or all of the savings too.

Agree about scale. Most software devs have no idea what can fit on a single server, and sometimes tend to start wanting complex scaling solutions for things where every possible customer they could ever get could fit on a single server

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#37

Post by zmmmmm »

piker wrote: post #33
zmmmmm wrote: post #29

Hope he's prepared for a letter from nVidia's lawyers for breaking their license agreement for the RTX3090's.

Is all commercial use of the consumer grade nVidia products prohibited under that license? I thought (and this seems to agree: https://www.nvidia.com/en-gb/drivers/geforce-license/ ) that it was just use in a datacenter that was prohibited.

I reckon they will consider what he has constructed to be a data center.

A lot of places do it and just fly under the radar, but if you're going to publish a blog post bragging about it and how much money you are saving ...

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#38

Post by Nextgrid »

anonymousDan wrote: post #32
lvl102 wrote: post #4

I hate the fact that hiring now basically requires cloud experience specific to vendors. This is basically going to force people into one of the three major cloud platforms.

For someone with a good general understanding of systems (distributed systems, operating systems etc), but perhaps lacking in hands on experience of deploying in a specific cloud, what would you say the key things are to learn? For me they almost seem like trivial things you could pick up in no time on the job.

Product-specific functionality & configuration. Every cloud provider has their own product, features & configuration for the same basic thing.

In the past, if you needed a load-balancer & reverse proxy you'd use Nginx or HAProxy regardless of the underlying machine. Now in the cloud, although you can technically run it on a VM, it's not "best practice" and you should instead reimplement it using your cloud vendor's proprietary equivalent, whether it's AWS ELB/ALB or something else, and that experience isn't portable across competing clouds.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#39

Post by Nextgrid »

bsenftner wrote: post #27
lvl102 wrote: post #4

I hate the fact that hiring now basically requires cloud experience specific to vendors. This is basically going to force people into one of the three major cloud platforms.

Unless, like OP, you work in a high compute field. High compute and the cloud simply do not mix, because the cloud is optimized for web apps, which require practically no real compute power. The cloud has it's purposes, as does the web, and they are not the only game in town.

Or high-bandwidth. Egress bandwidth charges on the cloud are robbery.

Re: We decided to move 90% of our workload from the cloud to on-prem infrastructure

#40

Post by Nextgrid »

recuter wrote: post #5
bob1029 wrote: post #2

> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Most don't. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per second is totally reasonable too if you are careful with a few things... I genuinely believe you could put the literal public Netflix biz experience on a single VM.…

Correctish in a sense: https://openconnect.netflix.com/en/appliances/#the-hardware They throw one of these boxes at an ISP and interconnect. 4-6 year no touch reliability, couple hundred TB storage. Modern hardware is quite something. They will saturate 2x100GE. That's in the thousands of concurrent streams per box.

> Modern hardware is quite something.

The one weird trick that cloud providers hate.

Post reply on HN