Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My VM is lighter (and safer) than your container (2017)

dl.acm.org

Post reply on HN
Print view View original

101–110 of 184 posts

Re: My VM is lighter (and safer) than your container (2017)

#101

Post by epr »

vegardx wrote: post #93
fhuici wrote: post #59

Earlier quoted context omitted.

And these overheads are even smaller if you use unikernels as per the paper. Eg, cold starts of a few milliseconds depending on the app/size of the image.

I'm struggling a little bit to grasp all the concepts when we start talking about unikernels, wasm and so on. Hopefully that's just a sign of the maturity of it, and not a sign of my mental decline. But on paper (as I understand it) it looks /so cool/.

Unikernels aren't too complicated conceptually. They're more or less a kernel stripped down to the bare minimum required by a single application. The complete bundle of the minimal kernel and application together is called a unikernel. The uni- prefix means one as in the kernel only supports one userspace application, instead of something like linux, which supports many. The benefits, as mentioned in the paper and in this thread are that you can run that as a vm, since it contains it's own operating system, unlike a container which is dependent on the host operating system. Also, they boot very quickly.

Re: My VM is lighter (and safer) than your container (2017)

#102

Post by pxc »

bartekrutkowski wrote: post #74
brabel wrote: post #13

Earlier quoted context omitted.

But if you can get isolation, security AND reproducible environments using a VM, specially one that's nearly as fast as a OS process, the case for using containers instead pretty much disappears. I don't know this LiteVM thing but I will definitely investigate that, specially given that on my Mac I need to use a VM anyway to run containers!

You don't need a VM on Mac to run containers, check out OrbStack, they provide a Docker compatible engine that is using native MacOS capabilities for running containers without the hidden Linux VM.

I don't know where you got that idea. OrbStack absolutely runs a Linux VM. That Linux VM then uses Linux containerization technologies (namely LXD) for each separate OrbStack 'machine' you set up, which is how you get such fast startup times for your OrbStack 'machines'.

For Docker, OrbStack does the same thing as Docker Desktop, Podman Desktop, Rancher Desktop, etc., which is set up a Linux VM running Docker and then present a native socket interface on macOS which relays everything it receives to the Docker socket inside the VM.

macOS doesn't have native capabilities for running containers, which is why the nearest thing you can get to containerd on it requires you to disable SIP so it can use a custom filesystem to emulate bind mounts/null mounts: https://darwin-containers.github.io/

If you read the PRs where the principal author of the Darwin Containers implementation is trying to upstream bits of his work, you'll see containerd comparing his approaches to others and complimenting them by calling them 'the most containerish' because real capabilities aren't there.

(I believe I've read rumors here on HN that Apple has those features internally, fwiw. But they've evidently never released them in a public copy of macOS.)

Another clue in all this is to just run uname in any of your Docker containers in OrbStack; you'll see they're Linux machines. Some operating systems have Linux syscall emulation layers (WSL1, FreeBSD's Linux emulation, Illumos' LX Zones) that could perhaps be used to run Linux containers without hardware emulation or paravirtualization in combination with some native containerization capabilities. Afaik Illumos' LX Zones is the only implementation where that's a supported, intended use case but maybe FreeBSD can do it. At any rate, macOS has never had that kind of syscall compatibility layer for Linux, either. So when you run `uname` in a 'macOS container' and see 'Linux', you can be certain that there's a VM in that stack.

PS: Aside from the fact that it's proprietary, I really do quite like OrbStack. It's the nicest-to-use implementation of something like this that I've tried, including WSL2 and Lima. The fact that it makes the VM machinery so invisible is very much to its credit from a UX perspective!

Re: My VM is lighter (and safer) than your container (2017)

#103

Post by andix »

dweekly wrote: post #15
andix wrote: post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

If this were true, then wouldn't folks just need an application binary that statically links all of its required libraries and resources into a giant, say, ELF? Why even bother with a container?

What if you application is not just one binary. What if it's a pipeline of complex tasks, calls some python scripts, uses a patched version of some obscure library, ...

It's not possible to package half a Linux distribution into a single binary. That's why we have containers.

Re: My VM is lighter (and safer) than your container (2017)

#104

Post by kylecordes »

fhuici wrote: post #96
zerof1l wrote: post #80

Docker has a big community, lots of guides, and ready-to-use containers. It became pretty much a de facto standard for self-hosting things. You also have a very high chance of getting a piece of software to work out of the box as intended with Doker. The only way this or some other way of running stuff will overtake Doker is if it will match the Docker in these aspects. As much as I'd love to try this lightweight VM…

Maybe try out kraft.cloud: we take Dockerfiles as input and automatically convert to lightweight VMs/unikernels when deploying (disclaimer: I'm one of the paper's authors and one of the people behind KraftCloud).

That is a very clever idea!

Re: My VM is lighter (and safer) than your container (2017)

#105

Post by fhuici »

jerf wrote: post #78
posix_monad wrote: post #50

Earlier quoted context omitted.

> build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against necessary bits of code It would be nice, but this is really hard to do when modern software has so many layers of crud. Good luck getting say, a PyTorch app, to work doing this without some serious time investment.

But you don't need to write against all the layers of crud. You only have to write against the bottom layer, the kernel API. This sort of software would have no need to specifically support "libxml" or "TLS", because that is multiple layers above what this sort of software does. The flip side is that if you want something like low-level access to your specific graphics card you may need to implement a lot of addition…

[Author of the paper here] You hit the nail on the head, this is precisely what we do (kernel API compatibility) with the LF Unikraft project (the evolution of the 2017 paper) at www.unikraft.org, and kraft.cloud, a cloud platform that leverages Unikraft.

Re: My VM is lighter (and safer) than your container (2017)

#106

Post by fhuici »

ascar wrote: post #63
bluGill wrote: post #57

Earlier quoted context omitted.

Most of that effort should be sharable. if you know you will only have one python process you can get rid of a lot of cruft. If you know you will be running in a VM then you only need the driver for the network interface the VM provides not every network interface every designed (often including ones that your hardware doesn't even physically support). So while there is serious time investment it isn't nearly as much…

I'm not sure if I missed a bit here, but I have some colleagues doing research on unikernels for HPC and the point is that this unikernel is running directly on the hardware or hypervisor and not inside another VM. The unikernel is effectively a minimal VM and the network stack is one of the things they struggle the most with due to sheer effort.

[One of the authors of the paper] I wouldn't recommend writing a network stack from scratch, that is a lot of effort. Instead, with the Unikraft LF project (www.unikraft.org) we took the lwip network stack and turned it into a Unikraft lib/module. At KraftCloud we also have a port of the FreeBSD stack.

Re: My VM is lighter (and safer) than your container (2017)

#107

Post by oceanplexian »

benreesman wrote: post #58
andix wrote: post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

Containers have their place, if you’re racking and running your own gear life gets a lot better than screwing around with IPMI. But reproducible infrastructure as code is just orthogonal to that: everything from Salt to Nix is credible in that role. Containerizing on top of a Xen hypervisor never made sense to me.

All the younger engineers I talk to think you would need to be Albert Einstein to bootstrap a bare metal server.

As someone who made a living doing this at scale, where we would build a new datacenter every 2-4 weeks using 100% open source or off the shelf tools, I completely disagree.

I think PXE booting some servers and running a binary on them is 90% easier than most container orchestration engines, Kubernetes control plane, and all the other problems engineers seem to have invented for themselves. I also think it’s almost always much more performant. Engineers don’t have an intuition to realize that their XXLarge-SuperDuper instance is actually a 5 year old Xeon they’re sharing with 4 other customers. Cloud Prociders obfuscate this as much as possible, and charge a King’s ransom if you want modern, dedicated hardware.

Re: My VM is lighter (and safer) than your container (2017)

#108

Post by BobbyTables2 »

zerof1l wrote: post #80

Docker has a big community, lots of guides, and ready-to-use containers. It became pretty much a de facto standard for self-hosting things. You also have a very high chance of getting a piece of software to work out of the box as intended with Doker. The only way this or some other way of running stuff will overtake Doker is if it will match the Docker in these aspects. As much as I'd love to try this lightweight VM…

It’s amazing what a few hundred million in VC funding can do!

Re: My VM is lighter (and safer) than your container (2017)

#109

Post by scarby2 »

cduzz wrote: post #97
andix wrote: post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

I tell people "An OCI container is a way to turn any random runtime into a statically linked binary." It is very useful for managing dependency hell, or at least moving it into "API dependencies" not "Library dependencies", it is handy for pickling a CI/CD release engineering infrastructure. It's not a security boundary. (I'm 100% agreeing with parent, in case I sound contentious)

> It's not a security boundary.

It is a security boundary, just not necessarily the best one.

Re: My VM is lighter (and safer) than your container (2017)

#110

Post by BobbyTables2 »

andix wrote: post #9

I think containers are often misunderstood: The main benefit is not isolation and security, it's defined and reproducible environments and builds. If there is some additional isolation required, just run the container in a VM.

I agree with you, but the world seems to think otherwise!
Post reply on HN