Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Does Developing on Kubernetes Suck?

blog.tilt.dev

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Why Does Developing on Kubernetes Suck?

#31

Post by the8472 »

hendry wrote: post #25
the8472 wrote: post #23

Earlier quoted context omitted.

If you have a fleet of services and currently work on services A and B you can just start up A - G in docker-compose based on CI-built images but mount your local build output into the A and B containers. Override their commands to watch filesystem changes so they reload themselves when you build. This can be easily managed with an personal override file on top of a committed compose file used by your team.

I agree the local dev environment UX is pretty much there. I use volume mounts myself. But when you're deploying on a remote host. OMG. Just sharing a Docker image internally is anguish. Developing as a remote team using Docker / Kubernetes workflow is just crazy. You're way WAY better off using serverless, where some implementations take ~2s to deploy my Go binary.

This can be hacked together in a similar way. Add a wrapper that watches for filesystem changes to the container command, then add a kubectl cp as last build step locally. Or push an image, set the deployment to pull on start and kill the pod.

For sharing docker images, assuming you have write access to some dev repository, you can push to a tag tied to your development branch and others can configure their docker setups to pull that on restart.

Dev images can be built faster by just copying build output into an image as last step, that way other steps to prepare the image are cached. No need to use whatever slow things CI is doing in its Dockerfile.

It only takes a few lines of shell or makefile to automate most of these things. Of course local dev is still nicer.

Re: Why Does Developing on Kubernetes Suck?

#32

Post by arpa »

dragonsh wrote: post #30
arpa wrote: post #2

Ok, wow. Talk about mixed feelings towards the article... the author is clearly knowledgeable, but it is so unclear why in the world does he opt for this kind of torture. 1. Why kubernetes for development? For most cases you don't need the whole orchestration over multiple nodes overhead, docker-compose is just fine. Incidentally, your push/pull problem is solved as you pull the dependencies and BUILD your app locall…

1. Kubernetes is when your service needs google kind of load which 90% of systems don't. Just trade it for simple VM or use LXD containers. Don't jump on next hype cycle. If something works for Google does not mean will work for you. 2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve prob…

for development, i agree. For production/testing/staging - hmmmm, it depends. I have taken a particular liking to the self-healing/HA properties of kubernetes. You just swap the dead nodes out without significant impact on the production... it's awesome.

Re: Why Does Developing on Kubernetes Suck?

#33

Post by nouney »

dragonsh wrote: post #30
arpa wrote: post #2

Ok, wow. Talk about mixed feelings towards the article... the author is clearly knowledgeable, but it is so unclear why in the world does he opt for this kind of torture. 1. Why kubernetes for development? For most cases you don't need the whole orchestration over multiple nodes overhead, docker-compose is just fine. Incidentally, your push/pull problem is solved as you pull the dependencies and BUILD your app locall…

1. Kubernetes is when your service needs google kind of load which 90% of systems don't. Just trade it for simple VM or use LXD containers. Don't jump on next hype cycle. If something works for Google does not mean will work for you. 2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve prob…

Managing a VM vs a managed Kubernetes cluster? I'd choose kubernetes without hesitating

Re: Why Does Developing on Kubernetes Suck?

#34

Post by arkh »

arpa wrote: post #2

Ok, wow. Talk about mixed feelings towards the article... the author is clearly knowledgeable, but it is so unclear why in the world does he opt for this kind of torture. 1. Why kubernetes for development? For most cases you don't need the whole orchestration over multiple nodes overhead, docker-compose is just fine. Incidentally, your push/pull problem is solved as you pull the dependencies and BUILD your app locall…

> docker-compose is just fine

Not when part of your dev use macOS and another part uses linux: gotta love how docker manages file permissions between host and containers.

Re: Why Does Developing on Kubernetes Suck?

#35

Post by arpa »

arkh wrote: post #34
arpa wrote: post #2

Ok, wow. Talk about mixed feelings towards the article... the author is clearly knowledgeable, but it is so unclear why in the world does he opt for this kind of torture. 1. Why kubernetes for development? For most cases you don't need the whole orchestration over multiple nodes overhead, docker-compose is just fine. Incidentally, your push/pull problem is solved as you pull the dependencies and BUILD your app locall…

> docker-compose is just fine Not when part of your dev use macOS and another part uses linux: gotta love how docker manages file permissions between host and containers.

docker on mac is a pain either way, since it sits in a linux vm and talks to mac fs via nfs. And as for permissions... well, it's been pretty much covered long time now - e.g. https://www.berthon.eu/2018/containers-volumes-and-file-perm...

hope this makes your life a bit easier.

Re: Why Does Developing on Kubernetes Suck?

#36

Post by mamon »

sp527 wrote: post #16

Local development on Kubernetes is in a pretty decent state imo. Minikube + Skaffold. Helm to manage/configure charts. And override resources to lower mem/cpu consumption when running locally. Works just fine for most situations. It's reproducible/consistent across developer environments and is as close as you can get to mapping 1-to-1 with prod.

So you install Minikube on each developer's machines? I thought that Minikube was created solely for running tutorials/excercises for people learning Kubernetes.

If you are developing microservices I think you're much better off with a development Kubernetes cluster, and a single namespace for each developer within it. This way the Kubernetes configuration can be identical to PROD, and you do not need tons of CPU cores and RAM on each of developer's laptops.

Re: Why Does Developing on Kubernetes Suck?

#37

Post by DonHopkins »

echopom wrote: post #4

> Why Does Developing on Kubernetes Suck ? IMHO because we are in a phase of transition. Having worked for years in software industry , I'm convinced we are halfway to a much bigger transformation for Software Engineers / SRE , Developers etc... I work in a Neobank ( N26 , Revolut, etc...) , we are currently in the process of re-writing our entire Core Banking System with MicroServices on top of Kubernetes with Kafka…

You need to upgrade to the new Zookeeper 2: Zookeepier!

https://www.youtube.com/watch?v=_F-RyuDLR4o

Re: Why Does Developing on Kubernetes Suck?

#38

Post by sametmax »

arpa wrote: post #32
dragonsh wrote: post #30

Earlier quoted context omitted.

1. Kubernetes is when your service needs google kind of load which 90% of systems don't. Just trade it for simple VM or use LXD containers. Don't jump on next hype cycle. If something works for Google does not mean will work for you. 2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve prob…

for development, i agree. For production/testing/staging - hmmmm, it depends. I have taken a particular liking to the self-healing/HA properties of kubernetes. You just swap the dead nodes out without significant impact on the production... it's awesome.

The trick is to understand that most products don't need to have 99% uptime. Your business will be fine if you are offline once in a while.

Hell, the stock market goes offline every night.

Those last % of uptime are very, very expensive at every level (money, people, logistic...). They better be worth it.

Re: Why Does Developing on Kubernetes Suck?

#39

Post by te_chris »

dragonsh wrote: post #30
arpa wrote: post #2

Ok, wow. Talk about mixed feelings towards the article... the author is clearly knowledgeable, but it is so unclear why in the world does he opt for this kind of torture. 1. Why kubernetes for development? For most cases you don't need the whole orchestration over multiple nodes overhead, docker-compose is just fine. Incidentally, your push/pull problem is solved as you pull the dependencies and BUILD your app locall…

1. Kubernetes is when your service needs google kind of load which 90% of systems don't. Just trade it for simple VM or use LXD containers. Don't jump on next hype cycle. If something works for Google does not mean will work for you. 2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve prob…

This misses the genius of k8s. When you adopt it, you’re adopting a completely declarative way of provisioning your system. It’s a total paradigm shift and I will never go back. My big want, is a kube as a service for small workloads - I.e I can just apply a deployment and everything goes up, no cluster considerations or whatever.

Re: Why Does Developing on Kubernetes Suck?

#40

Post by dragonsh »

nouney wrote: post #33
dragonsh wrote: post #30

Earlier quoted context omitted.

1. Kubernetes is when your service needs google kind of load which 90% of systems don't. Just trade it for simple VM or use LXD containers. Don't jump on next hype cycle. If something works for Google does not mean will work for you. 2. Don't spend too much on it just use traditional knowledge and, you can still use bare-metal, VM or LXD container. Focus on application not on programming a tool designed to solve prob…

Managing a VM vs a managed Kubernetes cluster? I'd choose kubernetes without hesitating

Don't manage VM, try LXD containers [1], they are containers with much better security and you can still use the same tools you learned for last decade or two to manage including the DSL if you like that.

Kubernetes is an Overkill for majority of development and production systems.

[1] https://linuxcontainers.org/

Post reply on HN