Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unfashionably secure: why we use isolated VMs

blog.thinkst.com

Post reply on HN
Print view View original

61–70 of 256 posts

Re: Unfashionably secure: why we use isolated VMs

#61

Post by topspin »

PedroBatista wrote: post #2

As a permanent "out of style" curmudgeon in the last ~15 years, I like that people are discovering that maybe VMs are in fact the best approach for a lot of workloads and the LXC cottage industry and Docker industrial complex that developed around solving problems created by themselves or solved decades ago might need to take a hike. Modern "containers" were invented to make things more reproducible ( check ) and sim…

Isn't this discussion based on a false dichotomy? I, too, use VMs to isolate customers, and I use containers within those VMs, either with or without k8s. These tools solve different problems. Containers solve software management, whereas VMs provide a high degree of isolation.

Container orchestration is where I see the great mistake in all of this. I consider everything running in a k8s cluster to be one "blast domain." Containers can be escaped. Faulty containers impact everyone relying on a cluster. Container orchestration is the thing I believe is "overused." It was designed to solve "hyper" scale problems, and it's being misused in far more modest use cases where VMs should prevail. I believe the existence of container orchestration and its misapplication has retarded the development of good VM tools: I dream of tools that create, deploy and manage entire VMs with the same ease as Docker, and that these tools have not matured and gained popularity because container orchestration is so easily misapplied.

Strongly disagree about containers and dev/deployment ("NOT check"). I can no longer imagine development without containers: it would be intolerable. Container repos are a godsend for deployment.

Re: Unfashionably secure: why we use isolated VMs

#62

Post by tptacek »

PedroBatista wrote: post #2

As a permanent "out of style" curmudgeon in the last ~15 years, I like that people are discovering that maybe VMs are in fact the best approach for a lot of workloads and the LXC cottage industry and Docker industrial complex that developed around solving problems created by themselves or solved decades ago might need to take a hike. Modern "containers" were invented to make things more reproducible ( check ) and sim…

Jails/Zones are not pretty much as secure as a VM. They're materially less secure: they leave cotenant workloads sharing a single kernel (not just the tiny slice of the kernel KVM manages). Most kernel LPEs are probably "Jail" escapes, and it's not feasible to filter them out with system call sandboxing, because LPEs occur in innocuous system calls, too.

Re: Unfashionably secure: why we use isolated VMs

#63

Post by tptacek »

nimish wrote: post #9
PedroBatista wrote: post #2

As a permanent "out of style" curmudgeon in the last ~15 years, I like that people are discovering that maybe VMs are in fact the best approach for a lot of workloads and the LXC cottage industry and Docker industrial complex that developed around solving problems created by themselves or solved decades ago might need to take a hike. Modern "containers" were invented to make things more reproducible ( check ) and sim…

Clear Containers/Kata Containers/firecracker VMs showed that there isn't really a dichotomy here. Why we aren't all using HW assisted containers is a mystery.

It's not at all mysterious: to run hardware-virtualized containers, you need your compute hosted on a platform that will allow KVM. That's a small, expensive, tenuously available subset of AWS, which is by far the dominant compute platform.

Re: Unfashionably secure: why we use isolated VMs

#64

Post by mwcampbell »

tptacek wrote: post #59

The cool kids have been combining containers and hardware virtualization for something like 10 years now (back to QEMU-Lite and kvmtool). Don't use containers if the abstraction gets in your way, of course, but if they work for you --- as a mechanism for packaging and shipping software and coordinating deployments --- there's no reason you need to roll all the way back to individually managed EC2 instances. A short s…

Since you're here, I was just thinking about how feasible it would be to run a microVM-per-tenant setup like this on Fly. I guess it would require some automation to create a Fly app for each customer. Is this something you all have thought about?

Re: Unfashionably secure: why we use isolated VMs

#65

Post by solarpunk »

reddit_clone wrote: post #47
analognoise wrote: post #28

Earlier quoted context omitted.

What do you think of Nix/NixOS?

But that comes _after_ you have chosen VMs over Containers yes? If you are using VMs, I think NixOs/Guix is a good choice. Reproducible builds, Immutable OS, Immutable binaries and Dead easy rollback. It still looks somewhat futuristic. Hopefully gets traction.

if you're using nixos, just to do provisioning, I would argue OStree is a better fit.

Re: Unfashionably secure: why we use isolated VMs

#66

Post by SunlitCat »

stacktrust wrote: post #48
SunlitCat wrote: post #46

VMs are awesome for what they can offer. Docker (and the like) are kinda a lean VM for a specific tool scenario. What I would like to see, would be more App virtualization software which isolates the app from the underlying OS enough to provide an safe enough cage for the app. I know there are some commercial offerings out there (and a free one), but maybe someone can chime in has some opinions about them or know som…

HP business PCs ship with SureClick based on OSS uXen, https://news.ycombinator.com/item?id=41071884

Thank you for sharing, didn't know that one!

Re: Unfashionably secure: why we use isolated VMs

#67

Post by the_duke »

ysnp wrote: post #27
compsciphd wrote: post #16

Earlier quoted context omitted.

As the person who created docker (well, before docker - see https://www.usenix.org/legacy/events/atc10/tech/full_papers/... and compare to docker), I argued that it wasn't just good for containers, but could be used to improve VM management as well (i.e. a single VM per running image - see https://www.usenix.org/legacy/events/lisa11/tech/full_papers... ) I then went onto built a system with kubernetes that enabled on…

Would you say approaches like gvisor or nabla containers provide more/enough evolution on the security front? Or is there something new on the horizon that excites you more as a prospect?

GVisor basically works by intercepting all Linux syscalls, and emulating a good chunk of the Linux kernel in userspace code. In theory this allows lowering the overhead per VM, and more fine-grained introspection and rate limiting / balancing across VMs, because not every VM needs to run it's own kernel that only interacts with the environment through hardware interfaces. Interaction happens through the Linux syscall ABI instead.

From an isolation perspective it's not more secure than a VM, but less, because GVisor needs to implement it's own security sandbox to isolate memory, networking, syscalls, etc, and still has to rely on the kernel for various things.

It's probably more secure than containers though, because the kernel abstraction layer is separate from the actual host kernel and runs in userspace - if you trust the implementation... using a memory-safe language helps there. (Go)

The increased introspectioncapabiltiy would make it easier to detect abuse and to limit available resources on a more fine-grained level though.

Note also that GVisor has quite a lot of overhead for syscalls, because they need to be piped through various abstraction layers.

Re: Unfashionably secure: why we use isolated VMs

#68

Post by bombela »

compsciphd wrote: post #16
PedroBatista wrote: post #2

As a permanent "out of style" curmudgeon in the last ~15 years, I like that people are discovering that maybe VMs are in fact the best approach for a lot of workloads and the LXC cottage industry and Docker industrial complex that developed around solving problems created by themselves or solved decades ago might need to take a hike. Modern "containers" were invented to make things more reproducible ( check ) and sim…

As the person who created docker (well, before docker - see https://www.usenix.org/legacy/events/atc10/tech/full_papers/... and compare to docker), I argued that it wasn't just good for containers, but could be used to improve VM management as well (i.e. a single VM per running image - see https://www.usenix.org/legacy/events/lisa11/tech/full_papers... ) I then went onto built a system with kubernetes that enabled on…

> As the person who created docker (well, before docker - see https://www.usenix.org/legacy/events/atc10/tech/full_papers/... and compare to docker)

I picked the name and wrote the first prototype (python2) of Docker in 2012. I had not read your document (dated 2010). I didn't really read English that well at the time, I probably wouldn't have been able to understand it anyways.

https://en.wikipedia.org/wiki/Multiple_discovery

More details for the curious: I wrote the design doc and implemented the prototype. But not in a vacuum. It was a lot work with Andrea, Jérôme and Gabriel. Ultimately, we all liked the name Docker. The prototype already had the notion of layers, lifetime management of containers and other fundamentals. It exposed an API (over TCP with zerorpc). We were working on container orchestration, and we needed a daemon to manage the life cycle of containers on every machine.

Re: Unfashionably secure: why we use isolated VMs

#69

Post by SunlitCat »

peddling-brink wrote: post #51
SunlitCat wrote: post #46

VMs are awesome for what they can offer. Docker (and the like) are kinda a lean VM for a specific tool scenario. What I would like to see, would be more App virtualization software which isolates the app from the underlying OS enough to provide an safe enough cage for the app. I know there are some commercial offerings out there (and a free one), but maybe someone can chime in has some opinions about them or know som…

That’s what containers attempt to do. But it’s not perfect. Adding a layer like gvisor helps, but again the app is still interacting with the host kernel so kernel exploits are still possible. What additional sandboxing are you thinking of?

Maybe I am a bit naive, but in my mind it's just a simple software running between the OS and the tool in question which runs said software in some kind of virtualization, passing all requests to the OS after a check what they might want to do.

I know that's what said tools are offering, but installing (and running) docker on Windows feels like loading up a whole other OS insides OS, so that even VM (Software) looks lean compared to that!

But I admit, that I have no real experience with docker and the like.

Re: Unfashionably secure: why we use isolated VMs

#70

Post by fsflover »

mikewarot wrote: post #39

It's nice to see the Principle Of Least Access (POLA) in practical use. Some day, we'll have operating systems that respect it as well. As more people wake up to the realization that we shouldn't trust code, I expect that the number of civilization wide outages will decrease. Working in the cloud, they're not going to be able to use my other favorite security tool, the data diode. Which can positively guarantee ingre…

> Some day, we'll have operating systems that respect it as well.

Qubes OS has been relying on it for many years. My daily driver, can't recommend it enough.

Post reply on HN