My VM is lighter (and safer) than your container (2017) • Hacker News Board

Re: My VM is lighter (and safer) than your container (2017)

#51

Post by convolvatron » Thu, Sep 08, 2022, 4:55 PM UTC

evol262 wrote: post #49

xani_ wrote: post #28

Earlier quoted context omitted.

They went to trash because containers are more convenient to use and saving few MBs of disk/memory is not what most users care. The whole idea was pretty much either use custom kernel (which inevitably have way less info on how to debug anything in it), and re-do all of the network and storage plumbing containers already do via the OS they are running one. OR just very slim linux one which at least people know how to…

Virtualization is not just "perceived" security over containerization. From CPU rings on down, it offers dramatically more isolation for security than containerization does. This isn't about 'what most users care' about either. Most users don't really care about 99% of what container orchestration platforms offer. The providers do absolutely care that malicious users cannot punch out to get a shell on an Azure AKS co…

in a related vein, most of the distinctions that are being brought up around containers vs vms (pricing, debugability, tooling, overhead) are nothing fundamental at all. they are both executable formats that cut at different layers, and there is really no reason why features of one can't be easily brought to the other.

operating above these abstractions can save us time, but please stop confusing the artifacts of implementation with some kind of fundamental truth. its really hindering our progress.

Re: My VM is lighter (and safer) than your container (2017)

#52

Post by nijave » Thu, Sep 08, 2022, 5:12 PM UTC

wyager wrote: post #40

It's not clear to me that VMs actually do offer better isolation than well-designed containers (i.e. not docker). It's basically a question of: do you trust the safety of kernel-mode drivers (for e.g. PV network devices or emulated hardware) for VMs, or do you trust the safety of userland APIs + the limited set of kernel APIs available to containers. On my FreeBSD server, I kind of trust jails with strict device rule…

I think it gets even more complicated with something like firecracker where they recommend you run firecracker in a jail (and provide a utility to set that up)

Re: My VM is lighter (and safer) than your container (2017)

#53

Post by nijave » Thu, Sep 08, 2022, 5:14 PM UTC

r3mc0 wrote: post #6

Containers and VMs are totally not the same thing. They serve a complete other purpose , as multiple containers can be combined to create an application/service , VMs always use a complete os etc etc anyway the internet is full of the true purpose of containers , they were never meant to use as a "VM" and about security.. meh everything is insecure until proven differently

VMs don't need a full OS. You can run a single process directly from the kernel with no init system or other userland

Re: My VM is lighter (and safer) than your container (2017)

#54

Post by discardedrefuse » Thu, Sep 08, 2022, 5:44 PM UTC

bongobingo1 wrote: post #13

I'm quite interested in seeing where slim VM's go. Personally I don't use Kubernetes, it just doesn't fit my client work which is nearly all single-server and it makes more sense to just run podman systemd units or docker-compose setups. So from that perspective, when I've peeked at firecracker, kata containers, etc, the "small dev dx" isn't quite there yet, or maybe never will get there since the players target othe…

> So from that perspective, when I've peeked at firecracker, kata containers, etc, the "small dev dx" isn't quite there yet, or maybe never will get there since the players target other spaces (aws, fly.io, etc). Stuff like a way to share volumes isn't supported, etc. Personally I find Dockers architecture a bit distasteful and Podmans tooling isn't quite there yet (but very close).

This is pretty much me and my homelab. I haven't visited it in a while, but Weave Ignite might be of interest here. https://github.com/weaveworks/ignite

Re: My VM is lighter (and safer) than your container (2017)

#55

Post by lamontcg » Thu, Sep 08, 2022, 5:45 PM UTC

Containers should really be viewed as an extension of packages (like RPM) with a bit of extra sauce with the layered filesystem, a chroot/jail and cgroups for some isolation between different software running on the same server.

Back in 2003 or so we tried doing this with microservices that didn't need an entire server with multiple different software teams running apps on the same physical image to try to avoid giving entire servers to teams that would be only using a few percent of the metal. This failed pretty quickly as software bugs would blow up the whole image and different software teams got really grounchy at each other. With containerization the chroot means that the software carries along all its own deps and the underlying server/metal image can be managed seperately, and the cgroups means that software groups are less likely to stomp on each other due to bugs.

This isn't a cloud model of course, it was all on-prem. I don't know how kubernetes works in the cloud where you can conceivably be running containers on metal sharing with other customers. I would tend to assume that under the covers those cloud vendors are using Containers on VMs on Metal to provide better security guarantees than just containers can offer.

Containers really shouldn't be viewed as competing with VMs in a strict XOR sense.

Re: My VM is lighter (and safer) than your container (2017)

#56

Post by hinkley » Thu, Sep 08, 2022, 6:03 PM UTC

All4All wrote: post #45

xahrepap wrote: post #41

This reminds me: in 2015 I went to Dockercon and one booth that was fun was VMWare's. Basically they had implemented the Docker APIs on top of VMWare so that they could build and deploy VMs using Dockerfiles, etc. I've casually searched for it in the past and it seems to not exist anymore. For me, one of the best parts of Docker is building a docker-image (and sharing how it was done via git). It would be cool to be…

Isn't that essentially what Vagrant and Vagrantfiles do?

What is your theory for why Docker won and Vagrant didn't?

Mine is that all of the previous options were too Turing Complete, while the Dockerfile format more closely follows the Principle of Least Power.

Power users always complain about how their awesome tool gets ignored while 'lesser' tools become popular. And then they put so much energy into apologizing for problems with the tool or deflecting by denigrating the people who complain. Maybe the problem isn't with 'everyone'. Maybe Power Users have control issues, and pandering to them is not a successful strategy.

Re: My VM is lighter (and safer) than your container (2017)

#57

Post by gavinray » Thu, Sep 08, 2022, 6:23 PM UTC

gamegoblin wrote: post #42

gavinray wrote: post #39

The issue with unikernels and things like Firecracker are that you can't run them on already-virtualized platforms I researched Firecracker when I was looking for an alternative to Docker for deploying FaaS functions on an OpenFaaS-like clone I was building It would have worked great if the target deployment was bare metal but if you're asking a user to deploy on IE an EC2 or Fargate or whatnot, you can't use these t…

This is a limitation of whatever virtualized instance you're running on, not Firecracker itself. Firecracker depends on KVM, and AWS EC2 virtualized instances don't enable KVM. But not all virtualized instance services disable KVM. Obviously, Firecracker being developed by AWS and AWS disabling KVM is not ideal :) Google Cloud, for instance, allows nested virtualization, IIRC.

Yeah but imagine trying to convince people to use an OSS tool where the catch is that you have to deploy it on special instances, only on providers that support nested virtualization

Not a great DX, haha I wound up using GraalVM's "Polyglot" abilities alongside it's WASM stuff

Re: My VM is lighter (and safer) than your container (2017)

#58

Post by faeriechangling » Thu, Sep 08, 2022, 6:49 PM UTC

What I see happening now on the cloud is containers from different companies and different security domain running on the same VM. I have to think this is fundamentally insecure and that VMs are underrated.

I hear people advocate QubesOS for security which is based on XEN when it comes to running my client. They say my banking should be done in a different VM than my email for instance. Well if that’s the case, why do we run many containers doing different security sensitive functions on the same VM when containers are not really considered a very good security boundary?

From a security design perspective I imagine hardware being exclusive to a person/organization, vms being exclusive to some security function, and containers existing on top of that makes more sense from a security function but we seem to be playing things more loosely on the server side.

Re: My VM is lighter (and safer) than your container (2017)

#59

Post by ksbrooksjr » Thu, Sep 08, 2022, 7:00 PM UTC

liftm wrote: post #11

1MachineElf wrote: post #7

Earlier quoted context omitted.

While a flawed comparison, WSL does use a VM in conjunction with the 9p protocol to achieve folder sharing.

9p-based folder sharing is (used to be?) possible with qemu, too.

It looks like it still is supported [1]. I noticed while reading the Lima documentation that they're planning on switching from SSHFS to 9P [2].

[1] https://wiki.qemu.org/Documentation/9psetup

[2] https://github.com/lima-vm/lima/blob/3401b97e602083cfc55b34e...

Re: My VM is lighter (and safer) than your container (2017)

#60

Post by duskwuff » Thu, Sep 08, 2022, 7:03 PM UTC

hinkley wrote: post #56

All4All wrote: post #45

Earlier quoted context omitted.

Isn't that essentially what Vagrant and Vagrantfiles do?

What is your theory for why Docker won and Vagrant didn't? Mine is that all of the previous options were too Turing Complete, while the Dockerfile format more closely follows the Principle of Least Power. Power users always complain about how their awesome tool gets ignored while 'lesser' tools become popular. And then they put so much energy into apologizing for problems with the tool or deflecting by denigrating th…

What turned me off from Vagrant was that Vagrant machines were never fully reproducible.

Docker took the approach of specifying images in terms of how to create them from scratch. Vagrant, on the other hand, took the approach of specifying certain details about a machine, then trying to apply changes to an existing machine to get it into the desired state. Since the Vagrantfile didn't (and couldn't) specify everything about that state, you'd inevitably end up with some drift as you applied changes to a machine over time -- a development team using Vagrant could often end up in situations where code behaved differently on two developers' machines because their respective Vagrant machines had gotten into different states.

It helped that Docker images can be used in production. Vagrant was only ever pitched as a solution for development; you'd be crazy to try to use it in production.