Adding Kubernetes support to the Docker platform • Hacker News Board

Re: Adding Kubernetes support to the Docker platform

#141

Post by holmb » Fri, Oct 20, 2017, 8:20 AM UTC

hosh wrote: post #98

bankss wrote: post #94

Earlier quoted context omitted.

Yeah, thanks for sharing your experiences on that. Unfortunately our egress costs would likely be significant. I am honestly anxious to get my employer started on the path to k8s, but until the tooling reduces the hours required to maintain it successfully, Swarm seems like the superior solution if you are locked in on a non-GKE/Azure Container Service provider. I have heard good things about Kops helping setup/enabl…

Digital Ocean recently announced a partnering with a Kubernetes vendor. You can run managed K8S on DO now.

Do you have a URL for that announcement?

Re: Adding Kubernetes support to the Docker platform

#142

Post by why-el » Sun, Oct 22, 2017, 11:27 PM UTC

InTheArena wrote: post #75

why-el wrote: post #71

Earlier quoted context omitted.

Not necessarily. It's like when Apple supports PowerPoint when it has Keynote.

Ehh. I think the writing is more then on the wall with Swarm. K8s is a better solution, and at this point. Docker + K8s should be the standard. Fragmentation is bad.

Yep, I agree with you.

Re: Adding Kubernetes support to the Docker platform

#143

Post by hosh » Mon, Oct 30, 2017, 8:20 PM UTC

avgkol wrote: post #139

hosh wrote: post #96

Earlier quoted context omitted.

We have less than 10 people, 5 of them engineers. We are overbooked getting features out. We are using K8S in both dev and production, not Swarm. A big part of that is because I have had experience running Docker (not Swarm), ECS, K8S, and building developer tooling (Vagrant), in addition to being a regular developer. The flip side: I had to opportunity to try out these different tech in production and saw where the…

Hosh , could you share your learning from setting up a prod K8 HA cluster? Could be useful for me.

I am not sure if I can. Setting it up by scratch let me become familiar with some of the underlying mechanisms of how k8s is put together. A part of that is:

https://rocketeer.be/blog/2015/11/kubernetes-from-the-ground...

And although I never ran through Hightower's Kubernetes the Hard Way, it is like that. https://github.com/kelseyhightower/kubernetes-the-hard-way

After running through that as a kind of kata, it was easier to infer and troubleshoot things when things go wrong. The transfer-of-learning happens only if you run yourself through these exercises.

I can share some things at a higher level though:

Label selectors are your friend. Master them. They are used everywhere.

Stateless is still easier than stateful. Start with putting stateless workloads in production before ever trying stateful.

If you have the expertise to mix your stateful pods with your stateless pods, make sure you master StatefulSet and things like persistant volume claims.

If you fake stateful pods like I did in production, then Kubernetes does not know how to cleanly shut them down. Automated maintenance involving kubectl cordon and drain no longer function well. You end up having to hand migrate stateful pods from node to node.

Re: Adding Kubernetes support to the Docker platform

#144

Post by hosh » Mon, Oct 30, 2017, 8:37 PM UTC

yumyumyumyum wrote: post #140

hosh wrote: post #95

Earlier quoted context omitted.

I see the same thing too: a struggle for relevance. The center of gravity has already moved onto k8s, even if it will takw a while for the rest of the developer community to catch up. There are a lot more exciting work around Helm (packaging for K8S), third party extensions (plugins for K8S), and Operators (embedded managed services). The K8S community already has more contributios going towards the stack, and the th…

From a production Swarm operator without much experience with k8s: When you create a new Swarm cluster with `docker swarm init`, Swarmkit creates PKI that is used for everything onward: gRPC cluster state communication, encrypted overlay networks, secrets, and so on. Keys are automatically rolled every 12 hours by default. This is done automatically and transparently to the operator, without any effort on their part.…

My team pays for GKE. The keys you are talking about are done with K8S + GKE so I don't see how that is much of an advantage other than cost. I want to add a new node, I go to the UI and increment a number.

As far as secrets go, there are a couple approaches the K8S community is trying for with encrypted secrets. The one I am rooting for is the integration with Hashicorp Vault. Chances are, that area will be fairly extensible.

I agree, 9k contributors and 9k pull requests are not to be scoff at. I wasn't scoffing at them. I'm simply saying that (1) center of gravity and where the influence is now has already shifted off of Docker, and (2) Just like Github.com phase-shifted open-source development (in way that Sourceforge never did), there are three technology in K8S that will likely phase-shift K8S development. This isn't about development power, but about a phase-shift.

I don't think I am being harsh on Swarm (and since when was this about being harsh? At the core of this is about whether the technology is fit-for-purpose and helps enable people achieve their mission and Docker is a commercial enterprise) The tipping point has come and gone. I saw Docker squander a lot of developer good-will over the past two years. I remember when CoreOS announced they were creating rkt, there were a lot of backlash to CoreOS. I liked CoreOS but I thought at the time it was a weird move. If that had happened this year, there would not be as much of a backlash.