Deploying Kubernetes with Ansible
erichreich.com
Deploying Kubernetes with Ansible
1–10 of 61 posts
Re: Deploying Kubernetes with Ansible
#2As your cluster grows, so does the complexity of managing it. That's where Ansible comes in.
Re: Deploying Kubernetes with Ansible
#3My conclusion is that using Ansible with Kubernetes is the WRONG way to go about it.
So now instead of a directory of yaml or helm charts we have a bunch of ansible uselessly wrapping the yaml for application.
Re: Deploying Kubernetes with Ansible
#4I think the lesson people need to start taking is that any form of declarative form of configuration will spawn an ad-hoc ecosystem to turn it imperative. The complexity of situations that can be expressed is by a configuration system is proportional to the complexity of that system. Simple and declarative is an oxymoron for anything but simple cases. Imperative systems can express arbitrary complexity and you will need it. It's why troposphere, terraform-cdk, pulumi, a billion templatating systems over k8s yaml files, and Ansible exist.
Re: Deploying Kubernetes with Ansible
#5My conclusion is that using Ansible with Kubernetes is the WRONG way to go about it. So now instead of a directory of yaml or helm charts we have a bunch of ansible uselessly wrapping the yaml for application.
The problem is when you're in a situation where "over here we need YAML document A, and over there we need YAML document B." You either shove a templating system in front of your YAML and fight with that or throw in the towel just use a programming language where constructs in your YAML map to objects.
Re: Deploying Kubernetes with Ansible
#6As your cluster grows, so does the complexity of managing it. That's where Ansible comes in.
I've never worked anywhere sizeable that uses ansible to manage kubernetes. My one brush with it was at a small shop and it caused more problems that it solved. It was just a couple of guys that happened to use ansible for everything else, right or wrong, it seemed like a lot of "i've got a hammer..."
In the pre-managed kubernetes world I did see an ansible stack for standing up a cluster that seemed ok, but once you get past setting up a base cluster ready to take 'customer' workloads ansible is just not a great tool.
Re: Deploying Kubernetes with Ansible
#7I thought this was about deploying a Kubernetes cluster with Ansible, but nope, it's really about deploying YAML files to Kubernetes, with Ansible.
Just...Why?
Re: Deploying Kubernetes with Ansible
#8Not really the best technical solution for this challenge. And this is from a big Ansible + Packer person that deploys 10Ks of VMs and infrastructure with Ansible.
Re: Deploying Kubernetes with Ansible
#9This went a different direction than I'd expected. That is, configuring your cluster resources rather than the host machines themselves.
I managed to grab 5 or so Raspberry Pi 4s before the availability issues. I've been (slowly) getting a home Kubernetes cluster running over the last year or two, and Ansible helps immensely; I:
* Write Ubuntu 20.04 (or now 22.04) to MicroSD card.
* Configure the initial user + SSH-PK, hostname, and network via config added to the SD
* Boot up the new Pi
* Run Ansible playbook to get it ready for k3s (lightweight but "full" k8s); including bind-mounting all high use paths for containerd/logs/etc to a USB NVME-SSD, adding boot args to the Pi's GRUB, etc.
* Push all my bashrc/tmux/vim/etc configs for consistent environment if I ssh in.
* Use Ansible ad-hoc for misc changes or package updates.
In terms of the "deploying to Kubernetes", I've recently got FluxCD working and love it. I think "GitOps" is the way to go in terms of clear reproducible config.
I have GitLab running on a local Synology (NAS) which kicks off any container builds and hosts the Git and container registry. The FluxCD operator on the cluster polls my "cluster as code" repo and syncs state. Each project defines a Helm chart and Flux pulls the chart from each repo and applies any variables (as configured in the "cluster repo") or "kustomizations".
Flux let's you setup "dependency" relationships for components. So I have it add the sealed-secrets operator early in, and put PK-encrypted secrets in directly the Git repo for it to"unpack" in the cluster.
That being said, the author's Anisble-driven approach is a much quicker way get something on a cluster ASAP, versus setting up the initial Flux "pull from cluster" system.
I'm primarily a software engineer, so this was also largely an exercise in getting more comfortable with the operations side of things.
Just thought I'd share my experience since only so many people let me rant about my FluxCD/k3s/GitLab home dev environment.
Re: Deploying Kubernetes with Ansible
#10I thought this was about deploying a Kubernetes cluster with Ansible, but nope, it's really about deploying YAML files to Kubernetes, with Ansible. Just...Why?
When all you have is a hammer, everything looks like a nail.