Kubernetes at GitHub

#1

Post by darwhy » Wed, Aug 16, 2017, 5:00 PM UTC

Kubernetes at GitHub
githubengineering.com

Re: Kubernetes at GitHub

#2

Post by DDub » Wed, Aug 16, 2017, 6:02 PM UTC

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

Re: Kubernetes at GitHub

#3

Post by MattRogish » Wed, Aug 16, 2017, 6:09 PM UTC

DDub wrote: post #2

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

We (ReactiveOps) use a combination of CircleCI and some scripts wrapping kubectl https://github.com/reactiveops/rok8s-scripts

Re: Kubernetes at GitHub

#4

Post by jameskilton » Wed, Aug 16, 2017, 6:17 PM UTC

DDub wrote: post #2

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

I've got a simple setup that makes use of YAML files, Rake tasks, and raw kubectl. I've yet to take a look at helm or spinnaker but it's on my list. You really can go a long way just with K8s' own tooling.

Re: Kubernetes at GitHub

#5

Post by jomakasi » Wed, Aug 16, 2017, 6:21 PM UTC

jameskilton wrote: post #4

DDub wrote: post #2

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

I've got a simple setup that makes use of YAML files, Rake tasks, and raw kubectl. I've yet to take a look at helm or spinnaker but it's on my list. You really can go a long way just with K8s' own tooling.

I've started with creating Rake tasks. Then I found https://github.com/CommercialTribe/psykube which is opinionated. Then I've ended up creating my own simple ruby tool to manage kubernetes with my own directory structure and configuration.

Re: Kubernetes at GitHub

#6

Post by foxylion » Wed, Aug 16, 2017, 6:29 PM UTC

DDub wrote: post #2

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

We also did some evaluation and then decided to stick to KISS an chose kubectl commands combined with cat and kexpand. Really simple approach to allow dynamic kubernetes deployments.

Example command can be

   cat service.yml | kexpand expand -v image-tag=git-135afed4 | kubectl apply -f -

The service.yml contains the full deployment configuration, service definition and ingress rules. So this works without preconfiguring anything in kubernetes when deploying an new service.

An engineer only has to create the service.yml and Jenkins does deploy it automatically on every master build.

*kexpand is a small tool which does something similar to sed, but in a simpler and less powerful way (keep it simple): https://github.com/kopeio/kexpand

Re: Kubernetes at GitHub

#7

Post by deadmik3 » Wed, Aug 16, 2017, 6:31 PM UTC

DDub wrote: post #2

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

I'd recommend looking into openshift. it's basically kubectl + cool deployment features. there's also free, paid, and dedicated online hosted options.

disclaimer: I work on openshift

Re: Kubernetes at GitHub

#8

Post by shock » Wed, Aug 16, 2017, 6:32 PM UTC

> During this migration, we encountered an issue that persists to this day: during times of high load and/or high rates of container churn, some of our Kubernetes nodes will kernel panic and reboot.

Considering that Kubernetes doesn't modify the kernel, this issue sounds like is present in mainline and kernel devs should be involved.

Re: Kubernetes at GitHub

#9

Post by Yeroc » Wed, Aug 16, 2017, 6:34 PM UTC

One thing I would have liked to have seen addressed in the article is whether the new architecture requires additional hardware (presumably) to operate and if so how much more.

Re: Kubernetes at GitHub

#10

Post by bryanlarsen » Wed, Aug 16, 2017, 6:37 PM UTC

DDub wrote: post #2

We're currently looking at moving our applications to k8s, and was wondering what deployment tools people are using? This week we are evaluating spinnaker, helm and bash wrappers for kubectl. There is concern over adding too many layers of abstraction and that KISS is the best approach.

We're using helm, but that was chosen mostly based on gut feel. It's an official project, and has momentum. We didn't want to spend too much time choosing a tool until we knew our requirements, and we don't really have a firm grasp of requirements until we've used something for a while. It seems to be working well for us so far, but it's still early.

There are lots of answers here that aren't helm, so I'm curious if there are any particular reasons that people ruled out helm?