Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Does Developing on Kubernetes Suck?

blog.tilt.dev

Post reply on HN
Print view View original

81–87 of 87 posts

Re: Why Does Developing on Kubernetes Suck?

#81

Post by nikisweeting »

imtringued wrote: post #46
echopom wrote: post #4

> Why Does Developing on Kubernetes Suck ? IMHO because we are in a phase of transition. Having worked for years in software industry , I'm convinced we are halfway to a much bigger transformation for Software Engineers / SRE , Developers etc... I work in a Neobank ( N26 , Revolut, etc...) , we are currently in the process of re-writing our entire Core Banking System with MicroServices on top of Kubernetes with Kafka…

>I'm sure this complexity is temporary. One tech will automate all of this away and building , deploying and running will be much simpler. Kubernetes was supposed to be one of those technologies.

> I'm convinced we are halfway to a much bigger transformation for Software Engineers / SRE , Developers etc...

I think kubernetes is the intro that gave everyone a taste of whats possible. What's yet to come is an easy abstraction layer or OS integration that makes it possible to use without learning 100 bits of new terminology and config minutiae.

Re: Why Does Developing on Kubernetes Suck?

#82

Post by ramanathanrv »

For small setups, it is better to stay away from Kubernetes until it reaches a good level of stability. We jumped early on in the K8S bandwagon (before AWS EKS) and paid a steep price for that. We continue to pay. Our AWS cost has doubled since the time we transitioned to K8S from AWS Elastic Beanstalk.

I really wish I could say what would be a good time to embrace K8S, but your mileage might vary. When we made the switch, our service was at about 80TPS during regular load and would go up to 300TPS during peak load.

Re: Why Does Developing on Kubernetes Suck?

#83

Post by atombender »

dragonsh wrote: post #30
arpa wrote: post #2

Ok, wow. Talk about mixed feelings towards the article... the author is clearly knowledgeable, but it is so unclear why in the world does he opt for this kind of torture. 1. Why kubernetes for development? For most cases you don't need the whole orchestration over multiple nodes overhead, docker-compose is just fine. Incidentally, your push/pull problem is solved as you pull the dependencies and BUILD your app locall…

1. Kubernetes is when your service needs google kind of load which 90% of systems don't. Just trade it for simple VM or use LXD containers. Don't jump on next hype cycle. If something works for Google does not mean will work for you. 2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve prob…

This meme needs to die. Kubernetes is not overkill for non-Google workloads.

In my current work, we run several Kubernetes clusters via GKE on Google Cloud Platform. We're a tiny company — less than 20 nodes running web apps, microservices and search engines — but we're benefiting hugely from the operational simplicity of Kubernetes. Much, much, much better than the old fleet of Puppet-managed VMs we used to run. Having surveyed the competition (Docker Swarm, Mesos/Marathon, Rancher, Nomad, LXD, etc.), I'm also confident that Kubernetes was the right choice.

Kubernetes may be a large and complex project, but the problem it solves is also complex. Its higher-level cluster primitives are vastly better adapted to modern operations than the "simple" Unix model of daemons and SSH and what not. The attraction isn't just the encapsulation that comes with containers, but the platform that virtualizes physical nodes and allows containers to be treated as ephemeral workloads, along with supporting primitives like persistent volumes, services, ingresses and secrets, and declarative rules like horizontal autoscalers and disruption budgets. Given this platform, you have a "serverless"-like magically scaling machine full of tools at your fingertips. You don't need a huge workload to benefit from that.

As for LXD and the other alternatives, these might be great, but Kubernetes has vastly more mainstream adoption these days. That's more important than mere technological superiority, or everyone would be using OpenSolaris with Zones and ZFS these days.

Re: Why Does Developing on Kubernetes Suck?

#84

Post by OJFord »

yongjik wrote: post #76
OJFord wrote: post #41

Earlier quoted context omitted.

> Kubernetes is when your service needs google kind of load which 90% of systems don't. This is the most frustrating but oft repeated nonsense about k8s. Kubernetes is an amazingly helpful abstraction for any amount of load on any larger than trivially tiny set of services/storage/etc. When you, like 90% of systems, don't have Google kind of load, you don't need Google number of nodes. Machine count corresponds to lo…

Kubernetes may be a useful framework, but "amazingly helpful" is not a word I'd use for it. To me, it's a half-assed collection of five hundred nearly identical (but all slightly different) yaml files masquerading as "abstraction". And because they're yaml you can't refactor out the common part. Well, to be fair, yaml itself doesn't exactly have a stellar reputation, but I've never seen fifty lines of yaml that could…

Use a Helm chart (or alternative)?

When you're writing YAML you can absolutely include from elsewhere (it calls it 'aliases', which is... Well, at least it exists) in multuple places - though annoyingly not from other files.

When you're reading (partially) generated YAML, (e.g. kubectl describe or edit) it's IMO a feature that everything is in its place with no indirection. (And as far as I'm aware this is YAML, not kubectl.)

Re: Why Does Developing on Kubernetes Suck?

#85

Post by ihcsim »

The project that I work on depends directly on the Kubernetes API, admission webhooks and API extensions. I do my daily development on a GKE cluster, because local k8s environments are slow and inconsistent. What I really like about tilt is that it enables the continuous development experience, where code changes are continuously built and deployed to my remote cluster as I made them in my editor. It helps to shorten my feedback loop by replacing the series of `docker build`, `docker push`, `kubectl apply` etc. commands with just a single `tilt up` command.

I can sympathize with what the author said. A while ago, I worked as the only devops engineer on a team of 10 full stack developers. The tl;dr is that management decided to migrate their existing workloads to k8s for reasons. The developers were not too keen about all the new concepts, tools and techniques they needed to learn as a result of that decision. They knew they had to own their code from dev to prod, and couldn't just throw it over the fence (because there was no fence). The issues that were brought up in the post definitely reasonated with some of the concerns the developers brought up. I do think a tool like tilt would have made the transition easier for that team.

Finally, I think the post could have been made simpler if it stays focus on just one tool, and how that tool helps to solve the problems that non-ops developers have, when writing code that needs to run on k8s.

Re: Why Does Developing on Kubernetes Suck?

#86

Post by longcommonname »

jasonvorhe wrote: post #9
echopom wrote: post #4

> Why Does Developing on Kubernetes Suck ? IMHO because we are in a phase of transition. Having worked for years in software industry , I'm convinced we are halfway to a much bigger transformation for Software Engineers / SRE , Developers etc... I work in a Neobank ( N26 , Revolut, etc...) , we are currently in the process of re-writing our entire Core Banking System with MicroServices on top of Kubernetes with Kafka…

Something is very wrong at this company if developers need hours or even days to deploy changes in a microservice architecture. I suggest to find an outside consultant with verifiable Kubernetes experience in production to get this checked out. Maybe a bit of outside perspective will help your teams to fix this. Good luck.

Legacy banking companies have very strict change management systems. These systems usually require many physical eyes on each change. This magnifests into a fear of fully automated change management.

Re: Why Does Developing on Kubernetes Suck?

#87

Post by pacala »

merb wrote: post #61
cbanek wrote: post #6

Earlier quoted context omitted.

I agree with you that many of these are self inflicted problems. I particularly like your answer to #3, which I think I'm going to have to start adopting! One part that I agree with the author is that local kubernetes clusters or even bare metal clusters are hard. I love being able to just spin up a cluster in GKE and have it just work. It really does just work, especially for ingress and storage. Get one piece worki…

> that local kubernetes clusters or even bare metal clusters are hard what's so hard about k3s, kubespray or minikube? the only problem k8s has, that without a linux dev machine you are going through the vm pain. but anything else is just simple. btw. using mac sucks when trying to develop with local k8s. windows is better/okish, but it's best to run directly on linux.

Year old experience: minikube requires root if you want to run directly on the host, without a VM layer. Every couple of months it goes into a borked state, which is nigh impossible to debug and sometimes hard to recreate from scratch, possibly due to the root issue above. Got a little luckier with Ubuntu and microk8.
Post reply on HN