If Docker Swarm is on the table (it's practically abandonware), I'd like to throw in Nomad by HashiCorp. It's also fairly lightweight, very flexible (can run various types of containers but also exec any executable), decent ecosystem (supports CNI and CSI). It can scale from single node control plane + workloads to tens of thousands of nodes, and the workers can also be geographically spread out (different regions/zo…
Oddly, I'm moving all my stuff to k8s because of how convenient and easy everything is to admin with tooling like k9s. But I've only ever worked with toy clusters with nodes in the single digits, and I can see what a nightmare debugging something like networking or storage at scale can be. Also, helm gave me a loathing for yaml I never knew I had in me. I avoid some of helm's more vulgar gyrations by using helmfile,…
If it's the whitespace that jams you up, I wanted to point out that since YAML is a superset of JSON, there's nothing stopping you from using text/template to cook JSON manifests. There is, of course, a toJson function available in helm https://helm.sh/docs/chart_template_guide/function_list/#typ...>
But I'm conceptually with you that the jokers who decided to use a text templating language for a structured output were gravely misguided and now we all suffer