Unfashionably secure: why we use isolated VMs

> This is wrong in pretty much every way I can imagine.

Nope! It’s not wrong in any way at all!

You’re thinking of how it’s built. I’m thinking of what it does (for me).

I tell it a package (image) to fetch, optionally at a version. It has a very large set of well maintained up-to-date packages (images). It’s built-in, I don’t even have to configure that part, though I can have it use other sources for packages if I want to. It fetches the package. If I want it to update it, I can have it do that too. Or uninstall it. Or roll back the version. I am 100% for-sure using it as a package manager, and it does that job well.

Then I run a service with a simple shell script (actually, I combine the fetching and running, but I’m highlighting the two separate roles it performs for me). It takes care of managing the process (image, which is really just a very-fat process for these purposes). It restarts it if it crashes, if I like. It auto-starts it when the machine reboots—all my services come back up on boot, and I’ve never touched systemd (which my Debian uses), Docker is my interface to that and I didn’t even have to configure it to do that part. I’m sure it’s doing systemd stuff under the hood, at least to bring the docker daemon up, but I’ve never touched that and it’s not my interface to managing my services. The docker command is. Do I see what’s running with systemd or ps? No, with docker. Start, restart, or stop a service? Docker.

I’ve been running hobbyist servers at home (and setting up and administrating “real” ones for work) since 2000 or so and this is the smoothest way to do it that I’ve seen, at least for the hobbyist side. Very nearly the only roles I’m using Docker to fill, in this scenario, are package manager and service manager.

I don’t care how it works—I know how, but the details don’t matter for my use case, just the outcomes. The outcome is that I have excellent, updated, official packages for way more services than are in the Debian repos, that leave my base system entirely alone and don’t meaningfully interact with it, with config that’s highly portable to any other distro, all managed with a common interface that would also be the same on any other distro. I don’t have to give any shits about my distro, no “oh if I want to run this I have to update the whole damn distro to a new major version or else manually install some newer libraries and hope that doesn’t break anything”, I just run packages (images) from Docker, update them with Docker, and run them with Docker. Docker is my UI for everything that matters except ZFS pool management.

> It's also not cross-platform, or at least 99.999% of images you might care about aren't— they're Linux-only.

I specifically wrote cross-distro for this reason.

> There actually are cross-platform package managers out there, too. Nix, Pkgsrc, Homebrew, etc.

Docker “packages” have a broader selection and better support than any of those, as far as services/daemons go; it’s guaranteed to keep everything away from the base system and tidy for better stability; and it provides a common interface for configuring where to put files & config for easier and more-confident backup.

I definitely use it mainly as a package manager and service manager, and find it better than any alternative for that role.

I see, you are unfamiliar with actual high security development and certification.

Common Criteria SKPP [1].

“AVA_VLA EXP.4.3E The NSA evaluator shall perform independent penetration testing.

AVA_VLA _EXP.4.4E The NSA evaluator shall determine that the TOE is resistant to penetration attacks performed by an attacker possessing a high attack potential.”

Which has been successfully certified against [2] for use in the F-22 and F-35.

Sneering dismissal of the existence of high security systems with actual empirical evidence while demanding recognition for unproven systems is ridiculous.

Oh, and by the way, the SKPP, like Level A1, required formal specifications and proofs. So, no, a demand to prove such systems are secure is not a winning move.

[1] https://www.commoncriteriaportal.org/nfs/ccpfiles/files/ppfi...

[2] https://www.commoncriteriaportal.org/nfs/ccpfiles/files/epfi...

Question: Could you get the customer isolation by running all console access through customer specific lambdas which simply add a unique (and secret) header to all requests. Then you can run a single database with sets of tables keyed by that secret header value.

Would give you very nearly as good isolation for much lower cost.

If you think about it virtualization is just a narrowing of the application-kernel interface. In a standard setting the application has a wide kernel interface available to it with dozens (ex. seccomp) to 100's of syscalls. A vulnerablility in any one of which could result in full system compromise.

With virtualization the attack surface is narrowed to pretty much just the virtualization interface.

The problem with current virtualization (or more specifically, the VMM's) is that it can be cumbersome, for example memory management is a serious annoyance. The kernel is built to hog memory for cache and etc. but you don't want the guest to be doing that - since you want to overcommit memory as guests will rarely use 100% of what is given to them (especially when the guest is just a jailed singular application), workarounds such as free page reporting and drop_caches hacks exist.

I would expect eventually to see high performance custom kernels for a application jails - for example: gVisor[1] acts as a syscall interceptor (and can use KVM too!) and a custom kernel. Or a modified linux kernel with patched pain points for the guest.

In effect what virtualization achieves is the ability to rollback much of the advantage of having an operating system in the first place in exchange for securely isolating the workload. But because the workload expects an underlying operating system to serve it, one has to be provided to it. So now you have a host operating system and a guest operating system and some narrow interface between the two to not be a complete clown show. As you grow the interface to properly slave the guest to the host to reduce resource consumption and gain more control you will eventually end up reimagining the operating system perhaps? Or come full circle to the BSD jail idea - imagine the host kernel having hooks into every guest kernel syscall, is this not a BSD jail with extra steps?

[1] https://gvisor.dev/>

> and Docker industrial complex that developed around solving problems created by themselves or solved decades ago.

From my perspective, it's the complete opposite: Docker is a workaround for problems created decades ago (e.g. dynamic linking), that could have been solved in a better manner, but were not.