>The need to configure, manage, monitor and maintain a whole extra set of programs. Also, systemd can just as easily start the binary in a cgroup, eliminating most of the following concerns.
In the former case, I need to start a (for example) Ubuntu image on my machine or VM. In the latter case, I need to start a Kubernetes image. It's not as different as you think. Kops can do the heavy lifting and even generate terraform manifests, which will put you far ahead of the manual solution in terms of controlling your infra.
Also, Cgroups alone aren't enough, and is only a small part of Docker's isolation, although I did actually acknowledge that Systemd supports them...
>Worth noting that behind the scenes, Linux is using the same in-memory versions of libc, even for containers. That isolation you speak of already doesn't exist.
No it doesn't. Assuming elf binaries, the elf binfmt will call into ld-linux shared object when you hit a dynamic binary, which then does the runtime linking that links libc dynamically. You can confirm that this will call the ld-linux from the container when in a container. Compare how printf acts inside of Alpine containers versus on its Debian host. Or try hosting a CGO_ENABLED=1 Go binary in a container with no libc - it won't load because the kernel won't find the linker. If you put it in an Alpine container and it was built on a Ubuntu host, you might find it cryptically fails to load!
>Ansible, Chef, Saltstack, etc.
This is not scheduling. That is automation, configuration. SaltStack does not calculate how much CPU and RAM is available on a box to balance applications that are running. Before I used Kubernetes, I used SaltStack to manage fat Go binaries and even Docker containers for a moment, so I have a pretty good idea of what the differences are.
>You have 65,000 ports available to you. I don't think this is as big an issue as made out to be - people are lazy, which is the only reason there are so many conflicts over 8000 and 8443.
Nothing to do with laziness. Why would I want to have to pick a new pprof port every single time then remember what app has which? Why would I want to then manage a firewall for all of that? If you're doing this all by hand eventually you're going to screw up. I'd rather have isolated networking deny-by-default.
>Given the context that I am someone currently attempting that hackery - this shit just kinda works, some of the time. And when it fails, it's a morass of conflicting documentation, and responses of "well, you're not using the default distribution for your environment" (no shit, it's no longer supported) and "works on my single node minikube cluster". Give me ansible/chef/saltstack any day of the week. Hell, I'll even take cfengine at this point.
That is an absolutely inaccurate characterization of the very professional and helpful Kubernetes community. I've only had a few run-ins with them, but the experiences were absolutely completely the opposite and I've literally never heard "works on my single node minikube cluster" and I doubt you have either.
>Kubernetes was built for Google's use case, in Google's environment. Attempting to use it with other restrictions and requirements is a nightmare.
And finally, no it wasn't. It was built by the Google Cloud Platform team to help Google Cloud Platform users better utilize their VMs. There's a pretty detailed history that's all publicly available. As far as we know, Google is yet to use Kubernetes itself, other than their spin-off using it for Pokemon Go. Google internally uses some progression of Borg.