Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A story about a Kubernetes migration

medium.com

Post reply on HN
Print view View original

31–40 of 60 posts

Re: A story about a Kubernetes migration

#31

Post by halbritt »

tekkk wrote: post #7

I hope that the original title of the story was intended sarcasm: "Unbabel migrated to Kubernetes and you won’t believe what happened next!" But so they managed to consolidate their infrastructure around Kubernetes and Google Cloud which made the management of their servers easier and faster? I wonder how much actual money they saved but I guess it will pay off for them in the long run. I've been dabbling with Kubern…

My org made a similar transition. It's hard to articulate exactly precisely how much we saved as our "production" or revenue generating environments are roughly the same.

We transitioned out of AWS where we had relatively well managed instances of our stack managed with chef and terraform to GKE in Google Cloud where we migrated to a helm chart and custom orchestration tooling on top of that.

Prior to the migration I'd say that 80% of our instances were idle. Currently, all of our k8s nodes with 16 cores are running with an average load 5-7. We try to keep enough headroom to prevent any waiting or queuing, which is an entire medium blog post unto itself.

So, roughly the same or a little more "production" workload, but the number of non-prod instances of our stack quadrupled. Anyone in the org at any time can spin up an instance of the stack for a custom sales demo, to debug an issue, to test a feature, or anything else. There was a great deal of pent up demand that nobody expected and which caused my team to thrash a bit to catch up to when we made the transition.

All in all, our GCP spend is about 20% less than our AWS spend was. We're getting a lot more utility for a little less money.

Re: A story about a Kubernetes migration

#33

Post by halbritt »

geekuillaume wrote: post #3

I'm working with Kubernetes recently and the learning curve is quite hard. I hope the team will improve kubectl to make it more user-friendly (error messages are hard to understand for beginners). A lot of cloud providers now have a way to easily deploy and manage a k8s cluster on their servers but I cannot find a tools that help with the deployment of a basic service, something like dokku but on Kubernetes. http://d…

Have you looked at helm?

Re: A story about a Kubernetes migration

#34

Post by halbritt »

shawabawa3 wrote: post #15
mosselman wrote: post #5

I am setting up a swarm deployment of one of my apps as an experiment and I must say the learning curve is hardly there. I tried kubernetes, but I found that most resources that try to explain how it works are focussing too much on github-size deployments. I just want 2 instances of my app, a database and traefik with lets encrypt. Does anyone know of a proper resource for the 'just a tad more than dokku' size?

Setting up a kubernetes cluster itself is probably the biggest hurdle. Also, bear in mind if it's just for a single service the resource overhead of kubernetes may be significant, possibly even more than 50%. I'd strongly recommend using a hosted k8s - either GKE, EKS, or I believe digital ocean have just released one. If you want to use an existing VPS just to test it out, see the docs here https://kubernetes.io/doc…

Triple-backing this recommendation.

My team has deployed in AWS using KOPS and EKS, Azure with AKS and ACS-engine, and GKE in GCP

GKE is far, far easier to manage, update, etc.

Re: A story about a Kubernetes migration

#35

Post by halbritt »

mosselman wrote: post #25
zerogvt wrote: post #17

Earlier quoted context omitted.

Maybe you can find a similar configuration as a helm chart ( https://helm.sh/ ) (i.e. a k8s pkg manager of sorts).

Yes, I saw helm and then I had to learn about yet another thing. Its all nice enough technology, don't get me wrong, but I think it isn't for 'running your own heroku' for your side projects. Maybe I am wrong though.

If what you want to run is already a helm chart, then that makes for a good starting point.

Re: A story about a Kubernetes migration

#37

Post by geerlingguy »

PedroArvela wrote: post #21
shawabawa3 wrote: post #16

Earlier quoted context omitted.

> k8s yaml files are really a breeze of fresh air compared to Ansible playbooks Hah. I'm a huge kubernetes fan but not sure I can agree here. k8s yaml files are the most verbose and spammy things imaginable. granted, ansible playbooks can be horrific, but i'd say that's more down to the authors of the playbook than ansible itself.

This is definitely a point. Kubernetes resource definitions are verbose, but you can expect them to always be about that verbose and nothing else. Ansible playbook instead really depend on the author, they can both be works of art or abominations.

I imagine as Kubernetes becomes more popular there will be a lot more of these abominations present... similar thing has happened in popular programming languages—as they are more widely adopted, early adopters who were more focused on quality and correctness are fewer, and new devs who do 'all the wrong things' are much more prevalent.

It's more of an issue with your organization's (or in some cases, personal) process if you allow abysmal code to get checked into your codebase :) Even Ansible has easy to integrate linting and testing tools.

Re: A story about a Kubernetes migration

#38

Post by geerlingguy »

SmirkingRevenge wrote: post #11
y4mi wrote: post #6

Earlier quoted context omitted.

Uuuh, seriously? I always preferred ansibles to kubernetes yaml I'm using both daily and can work with either though

Can't speak for the OP but I dislike the direction they seem to be heading, incrementally (and perhaps accidentally) - yaml as a Turing complete programming language.

> yaml as a Turing complete programming language.

If someone is authoring Ansible playbooks this way, this is definitely not a best practice. Code should go into modules, plugins, filters, etc. Playbooks should be YAML, with extremely minimal use of any coding constructs.

Re: A story about a Kubernetes migration

#40

Post by hinkley »

sandGorgon wrote: post #14
mosselman wrote: post #5

I am setting up a swarm deployment of one of my apps as an experiment and I must say the learning curve is hardly there. I tried kubernetes, but I found that most resources that try to explain how it works are focussing too much on github-size deployments. I just want 2 instances of my app, a database and traefik with lets encrypt. Does anyone know of a proper resource for the 'just a tad more than dokku' size?

Same here. I think if you have less than 100 servers, then k8s is a real overkill. Swarm is much easier to reason about and run. It's a godsend for startups without dedicated devops.

> dedicated devops.

I have dreamed a dream, but now that dream is gone from me.

Post reply on HN