Live data from Hacker News

My VM is lighter (and safer) than your container (2017)

dl.acm.org

41–50 of 184 posts

Re: My VM is lighter (and safer) than your container (2017)

#41
post #15
post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

If this were true, then wouldn't folks just need an application binary that statically links all of its required libraries and resources into a giant, say, ELF? Why even bother with a container?

glibc doesn't work statically linked and lots of stuff depends on glibc

Re: My VM is lighter (and safer) than your container (2017)

#42
post #4

From 2017, before rootless containers caught on I think. The conclusions on safety might be due for re-evaluating.

Why weren't containers rootless from the start anyway? What did they need that user space doesn't provide? Wine, emulators and VMs didn't require it either (with the exception of some VMs needing a kernel module for performance reasons like memory management, which I also find stupid, the OS should provide all the performance in user space).

Re: My VM is lighter (and safer) than your container (2017)

#43
post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

Containers are perfect for build environments and for creating the root filesystem. The issue is that the kernel these days are super bulky and are intended for multi-user, multi-process environments. Running a container runtime on top just makes it worse when you're looking for "isolation".

This paper argues that when you build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against necessary bits of code to execute _as_ a VM, then you'll get better performance than a container and you'll get that isolation.

This is in fact true based on performance studies, the follow up paper to this shows so: https://arxiv.org/pdf/2104.12721

(Disclosure, co-author of the linked paper.)

We ended up taking this to real workloads if you want to see it in action: https://unikraft.io/

Re: My VM is lighter (and safer) than your container (2017)

#44
post #4

From 2017, before rootless containers caught on I think. The conclusions on safety might be due for re-evaluating.

I don't really see how rootless containers change anything at all. You're still "just" one kernel privilege escalation away from breaking out. The level of isolation is much better in virtual machines, and the performance penalty is comparable these days.

The virtual machine images are a bit heavier, since you need a kernel and whatnot, but it's negligible at best. The memory footprint of virtual machines with memory deduplication and such means that you get very close to the footprint of containers. You have the cold start issue with microvms, but these days they generally start in less than a couple of hundred milliseconds, not that far off your typical container.

Re: My VM is lighter (and safer) than your container (2017)

#45

Earlier quoted context omitted.

Isolated, but are they isolated enough? The article states that containers offer weaker isolation than VMs. (it doesn't quantify it though and I don't know this kind of thing offhand)

Processes run in a userspace and cannot do anything without OS approval.

The Linux kernel has far too large of an attack surface to be trusted as a hard security boundary. It is good enough to prevent mostly trusted software from accidentally interfering with each other but I would not trust it to protect me from an untrusted workload.

For example GCP and AWS both have container running services. They both use hardware VMs to isolate different tenants. You will never share a kernel with another customer (I don't even think you will share one with yourself by default).

Re: My VM is lighter (and safer) than your container (2017)

#46
post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

[disclaimer: I'm one of the authors of the paper] I 100% agree, containers are an amazing dev env/reprodicble env tool! In fact, we think they're the perfect marriage to the unikernels (specialized VMs) we used in the paper; on kraft.cloud , a cloud platform we built, we use Dockerfiles to specify apps/filesystems, and transparently convert them to unikernels for deployment. The end result is the convenience of containers with the power of unikernels (eg, millisecond cold starts, scale to zero and autoscale, reduced TCB, etc).

Re: My VM is lighter (and safer) than your container (2017)

#47

"VM" means it has its own kernel? Why have 2 kernels on the same machine? All processes in a proper OS are already isolated and there is no need for VM.

Correct -- and you can run multiple kernels on the machine with virtualization extensions. Even Docker Desktop does this. You'd do this for _real_ isolation purposes.

Re: My VM is lighter (and safer) than your container (2017)

#48
post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

Containers are perfect for build environments and for creating the root filesystem. The issue is that the kernel these days are super bulky and are intended for multi-user, multi-process environments. Running a container runtime on top just makes it worse when you're looking for "isolation". This paper argues that when you build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against…

> Running a container runtime on top just makes it worse when you're looking for "isolation".

The point of the poster was pretty clear:

“The main benefit is not isolation and security”

Re: My VM is lighter (and safer) than your container (2017)

#49
post #13
post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

But if you can get isolation, security AND reproducible environments using a VM, specially one that's nearly as fast as a OS process, the case for using containers instead pretty much disappears. I don't know this LiteVM thing but I will definitely investigate that, specially given that on my Mac I need to use a VM anyway to run containers!

VMs in general use more CPU power as you have two OSes each doing things like updating their real time clock... There are VM aware OSes that will not do this, but it needs special code and CPU support which means you are often lagging behind the latest (to be fair this is rarely important) A container will normally be slightly faster than a VM never slower (assuming a reasonable OS - I can write an exception if I was malicious) and so there is a lot of interest if they are good enough.

Re: My VM is lighter (and safer) than your container (2017)

#50
post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

Containers are perfect for build environments and for creating the root filesystem. The issue is that the kernel these days are super bulky and are intended for multi-user, multi-process environments. Running a container runtime on top just makes it worse when you're looking for "isolation". This paper argues that when you build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against…

> build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against necessary bits of code

It would be nice, but this is really hard to do when modern software has so many layers of crud. Good luck getting say, a PyTorch app, to work doing this without some serious time investment.

Post reply on HN