What job interviews taught me about Kubernetes

I should add a disclaimer that I've never used kubernetes or docker, the latter mostly because I prefer the mental model and data encapsulation of a VM per service.

That said, the app I'm developing in my startup is designed with scalability from the outset. I have a single setup script for each type of node, and can take a fresh ubuntu install and just "wget -O- $URL | sh" as root and it sets up the node from scratch and/or reconfigures it to the latest configuration. That does all the ancillary stuff, setting up sane firewall defaults, blocking SSH from non-whitelisted IPs, setting up NTP, borg backup and zabbix (both require manual work on the respective backing servers currently), setting sane system configs (e.g. systemd logs limited to 100MB), wireguard (for the backends that distribute sqlite databases using litefs), etc. and installing the relevant packages with my software.

The actual backend application is built into a debian package automatically, so it's just a case of adding my private repository to apt sources and installing it. Updating a machine is just "ssh root@$MACHINE 'apt-get update ; apt-get install $APP'". I probably could automate that with ansible, but I prefer to upgrade them piecemeal while I'm testing out an upgrade, so I have a couple of bash scripts that do the ssh in a for loop instead with different targets in each.

This has the advantage for me of being able to buy any old VPS from a cheap provider and add it to my pool in minutes.

I'm sure I could end up with something that's just as easy to update with kubernetes, but it seems like another big learning curve with dependencies that probably change every few months and require me to keep learning new things just to keep it running. I understand my bash scripts, and know they won't just stop working going forwards (modulo exceptional events like having to migrate to systemd scripts, but that kind of change is usually only required on a very few major OS distribution upgrades).

I already have enough pain from some of my tech depending on other people's projects (I have a frontend app written in Flutter, and forced SDK upgrades about every 6 months and then resulting issues with toolchains I haven't even chosen to use, like gradle and kotlin, that seem to break everything every release), that I have no great desire to rebuild everything on someone else's deployment framework. When I get to the point of hiring others to help, I'd hope they'd be clued in enough to understand a simple bash script that sets up everything, and logically follow it through.

I'm not asking about kubernetes in interviews too, may be trying to understand do the interviewer understand how containerization, CNI, kube-proxy working under the hood. For me the best indicator of strong engineer is that he is interested enough to go under the hood, and understand the foundations rather the tools.

Docker swarm has been providing the simple solution for over a decade but nobody wants to use it because it's not k8s.

I’m personally a huge fan of k8s and while I agree it may be „complicated”, it’s because deploying applications is complicated. (I want to point out that there is no requirement no set up cert manager, ArgoCD, external secrets, etc. - and many people who’d consider a VPS would happily slap a .env with an unencrypted secret then ssh to update, but when they choose Kubernetes they take the long route of doing proper GitOps and complain that there are so many things to configure :)

I assumed the reason there are so many job offerings for kubernetes engineers is either because these jobs have high churn, or because kubernetes takes more work to maintain. I think linux+nginx+postgres+favourite-lang is still the most sensible way to go. Good engineers don't overcomplicate things.