Live data from Hacker News

Google admits Kubernetes container tech is too complex

theregister.com

401–410 of 449 posts

Re: Google admits Kubernetes container tech is too complex

#401
post #223

the pricing calculator does not work for autopilot which is really really sad. it always shows: Estimated Component Cost: per 1 month when configuring a pod with zero/1gb epheremal storage and 0.25 cpu and 500mib memory for 24/7.

Hi merb, it's William from Google.

If I enter the following, it seems to work for me: Replicas: 1, CPU: 0.25, Memory: 500MiB, Ephemeral Storage: 1 GiB.

The result is $9.92 per month for the us-central-1 location.

Re: Google admits Kubernetes container tech is too complex

#402

Kubernetes has to be most complex software I've ever tried to learn. I eventually gave up and decided to stick with simple single machine docker-compose deployments. I figure by the time any of my personal projects actually need to scale beyond 1 machine, I'd probably have enough revenue that I can afford to hire someone else to worry about it.

It's the right attitude. The management fees alone for this autopilot thing are 0.10$ per hour. Or about 70$/month. It's a bargain considering all the hidden costs kubernetes imposes in terms of requiring people that know how to tame the complexity associated with it (i.e. very expensive devops people costing magnitudes more than that). Automating those people away is worth money. I like Cloud Run for the same reason…

Hey, it's William from Google here. You're right about the costs, I just wanted to point out that Autopilot does include one cluster in GKE's free tier. So you'll only pay the ~$73/month if you have more than 1 cluster.

There's (almost) no limit to what you can run in one cluster too, and Kubernetes namespaces can help to separate different environments to allow for sharing.

Cloud Run sounds like the perfect solution for your workloads though!

Re: Google admits Kubernetes container tech is too complex

#403
In other news, water is wet.

Of course it's complex, it's managing a complex and very deep problem space through a mostly consistent generalized interface. Whether it is better than the alternatives depends on how deep your problems in that space go.

Re: Google admits Kubernetes container tech is too complex

#404

Earlier quoted context omitted.

Maybe your starting instances are workhorses, but if you go from something like a t3.medium to two it’s a ~$60 / mth increase... Not something I’d personally optimize for. Also why Docker in the first place? I’m genuinely wondering - in the stacks I run (Express / Python) it doesn’t seem necessary at low scale. Elastic Beanstalk, Heroku, Digital Ocean etc all offer facilities for single-command deploys that work out…

I use Docker without the network virtualization as a package manager. Docker make it easy to run the same version of code in different places and let’s things run next to each other without version conflicts. Also, I think you’re in a very small minority not to care about $720/yr increases in your hobbies.

GP was talking about projects that had revenue, and about "hiring someone" past a single instance.

I replied that beyond a single instance, you can probably get away with not hiring a K8s devops person and just spinning another instance. I'm not sure you've read this whole thing right.

And yes, I certainly wouldn't mind paying an additional $720 / yr for a project that had revenue; I almost certainly wouldn't want to spend money hiring a specialist, or spend time hyperoptimizing that myself - I make that in about a dozen hours of work, so counting how far one can go down the rabbit hole of optimizing server costs, and the associated cost of opportunity, the economics are crystal clear.

I don't have any successful personal projects but I have significant experience working with clients, and they are sold on the reasoning pretty much every single time ("I can charge you $3,000 for developing this feature, or we can use a paid service for $720 a year").

I also don't see how Docker is going to save you that much money; if you need a certain amount of compute, you need a certain amount of compute. AWS ElasticBeanstalk for instance charges nothing for spinning up an additional instance compared to EC2; there is no overhead for the PaaS aspect of it, like there would be in Heroku. Digital Ocean app platform is the same as EB, AFAIK.

Re: Google admits Kubernetes container tech is too complex

#405
post #77

Kubernetes has to be most complex software I've ever tried to learn. I eventually gave up and decided to stick with simple single machine docker-compose deployments. I figure by the time any of my personal projects actually need to scale beyond 1 machine, I'd probably have enough revenue that I can afford to hire someone else to worry about it.

Yes, k8s is not built for solo outfits and small projects. In places with 10+ developers and 10+ services on a single cluster it works surprisingly well from the user (developer) side.

In my opinion (and I'm biased as I work on GKE Autopilot), GKE is viable for a 1 developer project, especially with Autopilot mode (I have my own hobby projects deployed on GKE).

If you were self-hosting k8s, then I'd agree with you.

Re: Google admits Kubernetes container tech is too complex

#406
post #79

Earlier quoted context omitted.

Docker Swarm is pretty much abandonware/on life support at best, so one should avoid using it for new stuff. Hashicorp's Nomad is the best choice on the complexity for features scale IMHO, and that's why i'm writing an article how great it is, how easier some things are and what's missing compared to Kubernetes.

And yet, Docker Compose is pretty popular for local development, so much so, that it's not uncommon to find a docker-compose.yml in the repositories for many open source projects. And Docker Swarm builds on that, by bridging the gap between Docker Compose and multi-server deployments, with tools like Swarmpit and Podman for easier management of it as well, much like Rancher does for Kubernetes. I agree that Docker Sw…

> Nomad's read-only Web UI,

Nomad's UI has a good number of functionalities that can be controlled through it. Sure, there are some more lower level operations that are CLI only(though this seems to be something they are actively working to improve on) but most of that probably won't be needed but someone just trying to run a couple containers on a single node.

Re: Google admits Kubernetes container tech is too complex

#407

Earlier quoted context omitted.

I don’t know. I cant take anyone seriously who says it’s hard to type ./configure make At a prompt, then install missing libs. Unless you have to maintain updates regularly, “It’s just so hard” seems like a damn meme.

When library versions start to cause issues: like V3.5 having a bug, so you need to roll back to V3.4... that's when ./configure && make starts to have issues. Yeah, it happens with .so files, .dlls ("dll hell"), package managers and more. But that's where things like containers come in to help: "I tested Library Foo version V3.4 and that's what you get in the docker". No issues with Foo V3.5 or V3.6 causing issues..…

There are basically two options; maintain up-to-date dependencies carefully (engineer around dll-hell with lots of automated testing and be well-versed in the changelogs of dependencies) or compile a bunch of CVEs into production software.

There really isn't any middle ground (except to not use third-party libraries at all).

Re: Google admits Kubernetes container tech is too complex

#408
post #361

Earlier quoted context omitted.

> "Kubernetes is the biggest quality-of-life improvement I've experienced in my career." Just wait until the developers discover Google App Engine, Heroku, or DigitalOcean App Platform.

Yeah, I think thats the point though. Kubernetes enables the orchestration and observability of a PaaS in a much more flexible way so that you can get all of that while still matching the requirements of your business. I think Heroku and DigitalOcean App Platform are still going to be popular for small setups (as will things like Amplify) but when you outgrow those (or realize you are paying too much for them) then K…

+1. One benefit I see of Kubernetes is that it can handle pretty much whatever you throw at it, so you can run everything in the one service.

Need to run a bespoke database, yeah it can do that. Need to migrate an old service running in a VM that needs a disk, it can do that too.

Re: Google admits Kubernetes container tech is too complex

#409

I work on a 3-person DevOps team that just finished migrating ~20 services from GCE vms running docker-compose to GKE. It's taken us a little over a year. Partly because K8s has a steep learning curve, but also because safely transitioning services without disrupting product teams adds a lot of overhead. The investment is already yielding great returns. Developers are happy. Actual quote: "Kubernetes is the biggest q…

> ... That is until management decides we're overhead and fires us to free up budget for feature developers. ;)

It's a real risk, at least perceptually. Mitigate it by documenting the number of developers' hours you save with automation and better infrastructure. It's pretty hard to argue against a trend of increased developer productivity. Include your own hours; you're targeting saving 30 hours a week of your own time and that frees you up to improve other things even faster.

Re: Google admits Kubernetes container tech is too complex

#410
post #397
post #360

Earlier quoted context omitted.

Have you seen kops? rancher? loft? I don't think its that terrible to manage k8s at all using the tools available to you. k8s definitely used to be be difficult to manage but that isn't the case anymore. But literally every cloud provider out there has a managed solution so at this point you really only need to do it for DC work or if you like to do it. Amazon - EKS Google - GKE Azure - AKS anything beyond those 3 is…

I've used Rancher and while it is very nice it still punts a lot of the hard bits like interacting with the outside network, vm creation, and storage (especially shared persistent storage) to the ops team to figure out. Creating a big-ole cluster of app servers was never really the hard ops problem which is what k8s does really well.

Creating the cluster of servers isn't the value of kubernetes. We all had clusters of servers well before k8s, openstack, and things like AWS.

The benefit of k8s is the orchestration of those clusters. Spinning up 6 new http servers and getting them added to the load balancer automatically. Generating 4 new memcached nodes and getting them registered in DNS so clients pick them up and add them to the hashring.

The benefit of k8s is the scaling and elastic capabilities. It can trigger vertical scaling by spinning up larger pods or horizontal scaling by adding/removing pods.

Anyone thinking that people are using k8s because it can create app servers doesn't understand why anyone is using k8s. If all we needed to do is create a cluster of app servers, we wouldn't be using k8s.

That being said, a cluster of app servers still needs orchestration and config management and we had a ton of crazy solutions for that prior to k8s.

Post reply on HN