Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes a black hole of unpredictable spend, according to new report

theregister.com

Post reply on HN
Print view View original

41–50 of 88 posts

Re: Kubernetes a black hole of unpredictable spend, according to new report

#41

Post by cat199 »

kinghajj wrote: post #5
ferdowsi wrote: post #4

Earlier quoted context omitted.

From the application engineer side I'm not convinced that Kubernetes is particularly complex. I recently ramped up on it and found it liberating, frankly. Once I understood the basic concepts it was much more sensible than staring at a mountain of bespoke Ansible scripts operating on components I could barely see or understand. I can't speak to the SRE side; I can imagine the complexity there. But are these challenge…

"Any sufficiently complicated deployment and application management system contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Kubernetes."

... including kubernetes itself

Re: Kubernetes a black hole of unpredictable spend, according to new report

#42

Post by WYepQ4dNnG »

I have experienced first hand several cases of k8s gone wrong. In the end I have come to the conclusion that most companies don't really need the complexity of k8s.

Seriously, most k8s projects I have been involved with required so much effort to bootstrap and keep it going, it just blew me away! The experience for the average developer was just frustrating and infuriating: AWS ECS to the rescue!

Some will argue: vendor lock in! Really? I bet most services out there are already vendor locked in, just go with the flow and make your life easier.

I have seen companies failing because investing so much in building infrastructure, supposly vendor lock in free (or so they thought) that they lost sight and did not invest enough building the actual product: no revenue -> party is over.

Don't make the same mistake.

Re: Kubernetes a black hole of unpredictable spend, according to new report

#43

Post by jrockway »

avereveard wrote: post #37

Kubernetes will happily run without autoscaling. This article is barking at the wrong tree.

I am pretty sure that Kubernetes is just a buzzword they included to get clicks. You can definitely spend money in the cloud using Kubernetes or not using Kubernetes, and in the Kubernetes case, disabling node autoscaling doesn't eliminate the ability to spend money by submitting an API object. The article mentions PVs (which will be auto-provisioned whenever you request one; this feature is so core to Kubernetes that it works without effort even if you aren't using the cloud provider's managed Kubernetes offering) and cloud storage (write a byte, now you're spending money to store that byte), and it's right -- those things cost money and make spend unpredictable. But, it really has nothing to do with Kubernetes. They may have well as said "using Intel CPUs makes cloud spend unpredictable", because Intel CPUs are capable of executing instructions that call APIs to spend money in the cloud. Technically true, but kind of grasping at straws.

Re: Kubernetes a black hole of unpredictable spend, according to new report

#44

Post by imnotjames »

WYepQ4dNnG wrote: post #42

I have experienced first hand several cases of k8s gone wrong. In the end I have come to the conclusion that most companies don't really need the complexity of k8s. Seriously, most k8s projects I have been involved with required so much effort to bootstrap and keep it going, it just blew me away! The experience for the average developer was just frustrating and infuriating: AWS ECS to the rescue! Some will argue: ven…

ECS trades kubernetes manifests and generalized tools for cloudformation or the AWS UI, though..

Most everything else seems to be the same thing as if you ran with EKS, just different names for everything.

Setting it up yourself, though, no, I wouldn't do that unless I had a large enough team to maintain it.

Re: Kubernetes a black hole of unpredictable spend, according to new report

#46

Post by alongub »

I wrote a tool that helps estimate K8s costs by simulating K8s clusters. You write your pods in a simple DSL and it runs kube-scheduler without actual nodes behind the scenes.

It's still really basic but I'd love to hear your feedback!

https://github.com/aporia-ai/kubesurvival

Re: Kubernetes a black hole of unpredictable spend, according to new report

#47

Post by WYepQ4dNnG »

imnotjames wrote: post #44
WYepQ4dNnG wrote: post #42

I have experienced first hand several cases of k8s gone wrong. In the end I have come to the conclusion that most companies don't really need the complexity of k8s. Seriously, most k8s projects I have been involved with required so much effort to bootstrap and keep it going, it just blew me away! The experience for the average developer was just frustrating and infuriating: AWS ECS to the rescue! Some will argue: ven…

ECS trades kubernetes manifests and generalized tools for cloudformation or the AWS UI, though.. Most everything else seems to be the same thing as if you ran with EKS, just different names for everything. Setting it up yourself, though, no, I wouldn't do that unless I had a large enough team to maintain it.

> ECS trades kubernetes manifests and generalized tools for cloudformation or the AWS UI, though..

Terraform has served me well, it does come with some pain as well, but nothing compared to k8s.

Re: Kubernetes a black hole of unpredictable spend, according to new report

#48

Post by dan_quixote »

moksly wrote: post #2

I think the articles headline is a little rude to Kubernetes. I’m by no means a fan of Kubernetes, especially not in non-tech enterprise, but the article is really about the unpredictable and rising cost of moving into the cloud that is owned by the big tech companies, isn’t it? Sure kubernetes can be part of that, but you can easily run into the same predicament without it. The unpredictability of cost is actually t…

Are those of us happy/comfortable with current major cloud offerings simply not speaking up here? I can't fathom running a data center any longer for a company of nearly any size. Given my team's responsibilities, this would require 2-3x more headcount with significantly worse SLA/SLO if we still ran our own datacenter. Maybe it's not such a big deal for places with constant demand? Or is this just a case of observer bias?

Re: Kubernetes a black hole of unpredictable spend, according to new report

#49

Post by cobookman »

sleepybrett wrote: post #30
punkrex wrote: post #23

A lot of the “finops” practitioners I’ve seen are myopically focused on tagging of AWS resources; and that falls to pieces with kubernetes because AWS can’t see inside the kubernetes clusters. I’m not surprised they don’t like it.

I just had a meeting with a 'finops' manager where i showed him how kubernetes has a similar tagging structure (labels) and how we can break down per team pricing based on cpu/memory utilization. It's not hard, you just need the tools (kubecost, etc)

pod A uses 2 cores, pod B uses 1 core. Machine has 4 cores and all remaining unscheduled pods require 2 cores.

How do you attribute the partial usage of the node? Is it 2 cores billed to pod A, 1 core billed to pod B, and 1 core billed to some random team?

Or do you have 2/3 of Node billed to pod A and 1/3 of Node billed to Pod B.

Now deal with this permutation across all the various variables.

Re: Kubernetes a black hole of unpredictable spend, according to new report

#50

Post by Jochim »

bogota wrote: post #36
moksly wrote: post #2

I think the articles headline is a little rude to Kubernetes. I’m by no means a fan of Kubernetes, especially not in non-tech enterprise, but the article is really about the unpredictable and rising cost of moving into the cloud that is owned by the big tech companies, isn’t it? Sure kubernetes can be part of that, but you can easily run into the same predicament without it. The unpredictability of cost is actually t…

Unfortunately AWS is the new oracle. No one ever gets in trouble for picking it and its a great way to make it look like you as a high up exec provide value. Look how fast we are iterating now with my decision. It almost always ends in a mess of unmaintainable unthought out services that someone else has to come and clean up or move to the next proprietary service. The last 5 years for me has been soul crushing as so…

The problem with running your own DC is growing past your planned capacity. There's often a huge delay between developers having to put up with the VM infrastructure having to put up with under-resourced machines and more capacity being approved.

As a developer I've put up with over-subscribed VMware clouds and I vastly prefer the Azure/AWS option.

Post reply on HN