Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A story about a Kubernetes migration

medium.com

Post reply on HN
Print view View original

21–30 of 60 posts

Re: A story about a Kubernetes migration

#21

Post by PedroArvela »

shawabawa3 wrote: post #16
TeeWEE wrote: post #4

For me kubernetes is also a breeze. There is some learning curve because we started with Helm, Tiller, Grafana, Prometheus right from the start. But the kubectl command is easy to work with, and the k8s yaml files are really a breeze of fresh air compared to Ansible playbooks. We're not on production yet, but moving soon.

> k8s yaml files are really a breeze of fresh air compared to Ansible playbooks Hah. I'm a huge kubernetes fan but not sure I can agree here. k8s yaml files are the most verbose and spammy things imaginable. granted, ansible playbooks can be horrific, but i'd say that's more down to the authors of the playbook than ansible itself.

This is definitely a point.

Kubernetes resource definitions are verbose, but you can expect them to always be about that verbose and nothing else.

Ansible playbook instead really depend on the author, they can both be works of art or abominations.

Re: A story about a Kubernetes migration

#22

Post by PedroArvela »

shawabawa3 wrote: post #15
mosselman wrote: post #5

I am setting up a swarm deployment of one of my apps as an experiment and I must say the learning curve is hardly there. I tried kubernetes, but I found that most resources that try to explain how it works are focussing too much on github-size deployments. I just want 2 instances of my app, a database and traefik with lets encrypt. Does anyone know of a proper resource for the 'just a tad more than dokku' size?

Setting up a kubernetes cluster itself is probably the biggest hurdle. Also, bear in mind if it's just for a single service the resource overhead of kubernetes may be significant, possibly even more than 50%. I'd strongly recommend using a hosted k8s - either GKE, EKS, or I believe digital ocean have just released one. If you want to use an existing VPS just to test it out, see the docs here https://kubernetes.io/doc…

If you take the approach of a managed kubernetes instance, I really recommend GKE, everything is taken care of for you.

On the other hand EKS offers what I'd call a "managed kubernetes master", everything else is still pretty manual.

Re: A story about a Kubernetes migration

#23

Post by geggam »

zerogvt wrote: post #2

It seems that k8s has won the deployment race by and large. I see a lot of success stories around (I'm hearing nice things from the DevOps teams in my organization as well). Yet I'm curious to hear a few cases where things did not pan out quite right. Note: The 5-15s DNS problem seems a pretty serious one. Weird that it didn't get more publicity (and a proper fix).

Hard to fix something broken by design.

Using TCP as the main message bus then using layers upon layers of NAT needs to be revisited, routing is the solution.

Re: A story about a Kubernetes migration

#24

Post by zedpm »

It sounds like they really wanted to switch to K8s and rationalized it. The cons of their existing solution are minor and easily addressed with correct use of Ansible, and the massive complexity of K8s is understated.

As an example, they suggest that there's a heavy cognitive load associated with having devs run some Ansible playbooks, and then argue that to avoid that, they just have to introduce an entirely new toolchain via workshops and tutorials. Right.

Re: A story about a Kubernetes migration

#25

Post by mosselman »

zerogvt wrote: post #17
mosselman wrote: post #5

I am setting up a swarm deployment of one of my apps as an experiment and I must say the learning curve is hardly there. I tried kubernetes, but I found that most resources that try to explain how it works are focussing too much on github-size deployments. I just want 2 instances of my app, a database and traefik with lets encrypt. Does anyone know of a proper resource for the 'just a tad more than dokku' size?

Maybe you can find a similar configuration as a helm chart ( https://helm.sh/ ) (i.e. a k8s pkg manager of sorts).

Yes, I saw helm and then I had to learn about yet another thing. Its all nice enough technology, don't get me wrong, but I think it isn't for 'running your own heroku' for your side projects. Maybe I am wrong though.

Re: A story about a Kubernetes migration

#26

Post by mosselman »

sambull wrote: post #18
mosselman wrote: post #5

I am setting up a swarm deployment of one of my apps as an experiment and I must say the learning curve is hardly there. I tried kubernetes, but I found that most resources that try to explain how it works are focussing too much on github-size deployments. I just want 2 instances of my app, a database and traefik with lets encrypt. Does anyone know of a proper resource for the 'just a tad more than dokku' size?

Have you taken a look at Rancher [0]? Easy to spin up a couple nodes and manage them in a single place. It takes care of most of the issues I've found tedious for you with no issue. [0] https://rancher.com/

Thank you, but yes I have already tried Rancher. I have had it running with lets encrypt, etc. But it was pretty difficult to juggle all the different concepts: k8s has different concepts than rancher. What I also didn't like was when I had setup some things with the UI, I couldn't figure out how to create yamls for them.

Re: A story about a Kubernetes migration

#27

Post by mosselman »

sandGorgon wrote: post #14
mosselman wrote: post #5

I am setting up a swarm deployment of one of my apps as an experiment and I must say the learning curve is hardly there. I tried kubernetes, but I found that most resources that try to explain how it works are focussing too much on github-size deployments. I just want 2 instances of my app, a database and traefik with lets encrypt. Does anyone know of a proper resource for the 'just a tad more than dokku' size?

Same here. I think if you have less than 100 servers, then k8s is a real overkill. Swarm is much easier to reason about and run. It's a godsend for startups without dedicated devops.

Swarm is working out really nicely so far.

Re: A story about a Kubernetes migration

#29

Post by falcolas »

eeZah7Ux wrote: post #20
falcolas wrote: post #12

Earlier quoted context omitted.

Saying that Kubernetes is a bit complicated seems like saying that water can be a bit wet. Even their documentation can't keep up. And with a release cycle of 3 months, and a deprecation cycle of 6 months, you need a team dedicated to keeping up with K8s state-of-the-art; so much of that knowledge you picked up a year ago is at best stale, and at worst wrong. Sure, it makes setting up and keeping a set of containers…

"Now we have a distributed monolith that requires 2x less developers to build and 5x more system engineers to deploy"

Well, to be fair, all of those developers probably found themselves filling a systems engineer role "because the product developers are best equipped to handle the running and support of their own applications".

Re: A story about a Kubernetes migration

#30

Post by halbritt »

zedpm wrote: post #24

It sounds like they really wanted to switch to K8s and rationalized it. The cons of their existing solution are minor and easily addressed with correct use of Ansible, and the massive complexity of K8s is understated. As an example, they suggest that there's a heavy cognitive load associated with having devs run some Ansible playbooks, and then argue that to avoid that, they just have to introduce an entirely new too…

Regardless of your skepticism, the benefits are real.

Scaling applications in k8s, updating, and keeping configs consistent are a great deal easier for me than using Ansible or any other config management tool.

In the end, it's a singular platform that one can build tooling against that allows an organization to abstract away the infrastructure. My team has done that (on top of k8s). As such, a developer can spin up a new environment with the click of a button, deploy whatever code they like, scale the environment, etc. with very little to no training. Those capabilities were a tremendous accelerator for my organization.

Sure, you can build something similar with Ansible on AWS, but then you're married to AWS, you have to worry about sizing, and the cost of idle instances. In my experience, it's just a great deal more overhead.

Post reply on HN