Ask HN: Docker, Kubernetes, Openshift, etc – how do you deploy your products?

I've worked with a self hosted Gitlab instance before and that was nice, but I'm not quite sure how well the Gitlab CI would integrate when the code is on github.

Back in 2015, I implemented a Kubernetes by hand in AWS. I'm not going to do something like that again. GKE is fairly painless and it has most of the sensible defaults that I want. Networking just works -- pods can talk to each other as well as to any VM instances from any availability zone and region. Integrating with GCP service accounts just works. Spinning up experimental clusters is easy, as is horizontally scaling the clusters. One gotcha is that Google has not made K8S 1.5 generally available in all regions or availability zones. Otherwise, upgrades are pretty easy.

I have deployed with Docker Compose (not doing that again -- it is easier to use shell scripts). I have deployed with AWS ECS service (not doing that again; it does not have the concept of pods which severely constrains how you deploy). I used to deploy with Chef. I've heard of Chef's Habitat, but have not played with it.

Back for the 2015 project, I wrote Matsuri as a framework to manage the different Kubernetes templates. It's useful if you know Ruby. It uses idiomatic Ruby to generate and manage K8S specifications, and run kubectl commands. I wanted a single tool that could work with all the different environments (production, staging, etc.) as well as manage the dev environment. For example, if I want to diff my version-controlled spec on dev with what Kubernetes master currently has, I would use `bin/dev diff pod myapp`. If I want to diff the deployment resource by the same name, I would use `bin/production diff deployment myapp`. I can write hooks specific to the app. For example, `bin/production console mongodb` uses hooks to query Kubernetes to find a pod to attach to, determine the current Mongodb master, and invoke the command to go directly into the Mongodb shell. But I could have invoked `bin/staging console mongodb` or `bin/dev console mongodb`. I could do this because I have been developing software for a long time and I have enough ops experience to be able to put it all together. YMMV.

We're using Go.cd for the CD. I could have used Jenkins, but decided to give Go.cd a try. Go.cd has some advantages (such as much better topologies and tracking value streams) though there are also things it does not do as well as Jenkins (Go.cd auth mechanisms blow, and I had to write my own custom proxy to get Github hooks working more securely and reliably). Setting up GCP service accounts so that go.cd agents can deploy was a lot easier than I thought, once I read through the GCP docs. (Much easier than AWS).

Docker containers are still difficult to make. You want to vet things before using them. Handling this stuff is still going to be a full-time job for someone, both in terms of designing the infrastructure as well as the development tools. There are a lot of issues that come up because dev might throw things over the wall that might impact the overall reliability and performance of the system.

Do you have large workload spikes, or traffic spikes?

Do you have any advice, or gotchas on getting into kubernetes development?

The build jobs creates images that are published to ECS repositories, and there are auto scaling groups that add and remove engine hosts to and from ALB target groups for each deployed service. It makes service discovery, scaling, etc. really easy.

Definitely try swarm out if you haven't already. 1.12 was good, 1.13 is amazing (secrets, health-based VIP membership, etc).