Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unfashionably secure: why we use isolated VMs

blog.thinkst.com

Post reply on HN
Print view View original

91–100 of 256 posts

Re: Unfashionably secure: why we use isolated VMs

#91

Post by tptacek »

Veserv wrote: post #87
fsflover wrote: post #72

Earlier quoted context omitted.

Show me a recent escape from VT-d and then you will have a point.

VT-x. You should get the name of the technology right before defending it. VT-d is the I/O virtualization technology. When did it become customary to defend people making claims of security instead of laughing in their face even though history shows them such claims to be a endless clown parade? How about you present the extraordinary evidence needed to support the extraordinary claim that there are no vulnerabilitie…

Not an especially impressive flex, but I'm not above trying to dunk on people for misspelling things either, so I'm not going to high-horse you about it (obviously i am).

The history of KVM and hardware virtualization is not an endless clown parade.

Find a vulnerability researcher to talk to about OpenBSD sometime, though.

https://isopenbsdsecu.re/

Re: Unfashionably secure: why we use isolated VMs

#92

Post by compsciphd »

bombela wrote: post #68
compsciphd wrote: post #16

Earlier quoted context omitted.

As the person who created docker (well, before docker - see https://www.usenix.org/legacy/events/atc10/tech/full_papers/... and compare to docker), I argued that it wasn't just good for containers, but could be used to improve VM management as well (i.e. a single VM per running image - see https://www.usenix.org/legacy/events/lisa11/tech/full_papers... ) I then went onto built a system with kubernetes that enabled on…

> As the person who created docker (well, before docker - see https://www.usenix.org/legacy/events/atc10/tech/full_papers/ ... and compare to docker) I picked the name and wrote the first prototype (python2) of Docker in 2012. I had not read your document (dated 2010). I didn't really read English that well at the time, I probably wouldn't have been able to understand it anyways. https://en.wikipedia.org/wiki/Multipl…

I'd note I didn't say you copied it, just that I created it first (i.e. "compare paper to docker". also, as you note, its possible someone else did it too, but at least my conception got through academic peer-review / patent office, yeah, there's a patent, never been attempted to be enforced though to my knowledge).

when I describe my work (I actually should have used quotes here), I generally give air quotes when saying it, or say "proto docker", as it provides context for what I did (there's also a lot of people who view docker as synonymous with containerization as a whole, and I say that containers existed way before me). I generally try to approach it humbly, but I am proud that I predicted and built what the industry seemingly needed (or at least is heavily using).

people have asked me why I didn't pursue it as a company, and my answer is a) I'm not much of an entrepreneur (main answer), and b) I felt it was a feature, not a "product", and would therefore only really profitable for those that had a product that could use it as a feature (which one could argue that product turned out to be clouds, i.e. they are the ones really making money off this feature). or as someone once said a feature isn't necessarily a product and a product isn't necessarily a company.

Re: Unfashionably secure: why we use isolated VMs

#93

Post by tptacek »

Spivak wrote: post #90
tptacek wrote: post #63

Earlier quoted context omitted.

It's not at all mysterious: to run hardware-virtualized containers, you need your compute hosted on a platform that will allow KVM. That's a small, expensive, tenuously available subset of AWS, which is by far the dominant compute platform.

So… Lambda, Fargate, and EC2. The only thing you can't really do this with is EKS. Like Firecracker was made by AWS to run containers on their global scale KVM, EC2.

Lambda and Fargate are implementations of the idea, not a way for you yourself to do any kind of KVM container provisioning. You can't generally do this on EC2; you need special instances for it.

For a variety of reasons, I'm pretty familiar with Firecracker.

Re: Unfashionably secure: why we use isolated VMs

#94

Post by xtacy »

compsciphd wrote: post #83
the_duke wrote: post #67

Earlier quoted context omitted.

GVisor basically works by intercepting all Linux syscalls, and emulating a good chunk of the Linux kernel in userspace code. In theory this allows lowering the overhead per VM, and more fine-grained introspection and rate limiting / balancing across VMs, because not every VM needs to run it's own kernel that only interacts with the environment through hardware interfaces. Interaction happens through the Linux syscall…

I actually wonder how much "overhead" a VM actually has. i.e. a linux kernel that doesn't do anything (say perhaps just boots to an init that mounts proc and every n seconds read in/prints out /proc/meminfo) how much memory would the kernel actually be using? So if processes in gvisor map to processes on the underlying kernel, I'd agree it gives one a better ability to introspect (at least in an easy manner). It give…

> to have a tool that is external to the VM (runs on the hypervisor host) that essentially has "read only" access to the kernel running on the VM to provide visibility into what's running on the machine without an agent running within the VM itself

Not quite what you are after, but comes close ... you could run gdb on the kernel in this fashion and inspect, pause, step through kernel code: https://stackoverflow.com/questions/11408041/how-to-debug-th....

Re: Unfashionably secure: why we use isolated VMs

#95

Post by compsciphd »

tptacek wrote: post #86
compsciphd wrote: post #38

Earlier quoted context omitted.

been out of the space for a bit (though interviewing again, so might get back into it), gvisor at least as the "userspace" hypervisor, seemed to provide minimal value vs modern hypervisor systems with low overhead / quick boot VMs (ala firecracker). With that said, I only looked at it years ago, so I could very well be out of date on it. Wasn't aware of Nabla, but they seem to be going with the unikernel approach (ba…

Modern gVisor uses KVM, not ptrace, for this reason.

so I did a check, it would seem that gvisor with kvm, mostly works for bare metal, not on existing VMs (nested virtualization).

https://gvisor.dev/docs/architecture_guide/platforms/

"Note that while running within a nested VM is feasible with the KVM platform, the systrap platform will often provide better performance in such a setup, due to the overhead of nested virtualization."

I'd argue then for most people (unless have your own baremetal hyperscaler farm), one would end up using gvisor without kvm, but speaking from a place of ignorance here, so feel free to correct me.

Re: Unfashionably secure: why we use isolated VMs

#96

Post by Veserv »

tptacek wrote: post #71
Veserv wrote: post #33

Earlier quoted context omitted.

[flagged]

This Theo quote from 18 years ago gets brought up a lot. It's referring to a different era in virtualization (it practically predates KVM, and certainly widespread use of KVM). You can more or less assume he's talking about running things under VMWare. In the interim: * The Linux virtualization interface has been standardized --- everything uses the same small KVM interface * Security research matured and, in particu…

Are you claiming it has no security vulnerabilities? If yes, care to present a proof. If no, then please estimate how big of a bug bounty would result in a reported critical vulnerability.

If I put up a 1 G$ bug bounty, do you think somebody would be able to claim it within a year? How about 10 M$? Please justify this in light of Google only offering 250 k$ [1] for a vulnerability that would totally compromise the security foundation of the multi-billion (trillion?) dollar Google Cloud.

Please also justify why the number you present is adequate for securing the foundation of the multi-trillion dollar cloud industry. I will accept that element on its face if you say the cost would be 10 G$, but then I will demand basic proof such as formal proofs of correctness.

[1] https://security.googleblog.com/2024/06/virtual-escape-real-...

Re: Unfashionably secure: why we use isolated VMs

#97

Post by tptacek »

Veserv wrote: post #96
tptacek wrote: post #71

Earlier quoted context omitted.

This Theo quote from 18 years ago gets brought up a lot. It's referring to a different era in virtualization (it practically predates KVM, and certainly widespread use of KVM). You can more or less assume he's talking about running things under VMWare. In the interim: * The Linux virtualization interface has been standardized --- everything uses the same small KVM interface * Security research matured and, in particu…

Are you claiming it has no security vulnerabilities? If yes, care to present a proof. If no, then please estimate how big of a bug bounty would result in a reported critical vulnerability. If I put up a 1 G$ bug bounty, do you think somebody would be able to claim it within a year? How about 10 M$? Please justify this in light of Google only offering 250 k$ [1] for a vulnerability that would totally compromise the se…

I have no idea who you're talking to, but nobody on this thread has claimed anything has "no security vulnerabilities". If you think there isn't an implicit 7-figure bounty on KVM escapes, we are operating from premises too far apart for further discussion to be productive.

My bigger problem though: I gave you a bunch of substantive, axiomatic arguments, and you responded to none of them. Of the three of them, which were you already aware of? How did your opinion change after learning about the other ones? You cited a 2007 Theo argument in 2024, so I'm going to have trouble with the idea that you were aware of all of them; again, I think even Theo would be correcting your original post.

later

You've written about the vulnerability brokers you know in other posts here; I assume we can just have a substantive, systems based debate about this claim, without needing to cite Theo or Joanna Rutkowska or whatever.

Re: Unfashionably secure: why we use isolated VMs

#98

Post by jinzo »

er4hn wrote: post #82

One thing I wasn't able to grok from the article is orchestration of VMs. Are they using AWS to manage the VM lifecycles, restart them, etc? Last time I looked into this for on-prem the solutions seemed very enterprise, pay the big bux, focused. Not a lot in the OSS space. What do people use for on-prem VM orchestration that is OSS?

Depends what is your scale, but I used oVirt and Proxmox in the past, and it was (especially oVirt) very enterprisey but OSS.

Re: Unfashionably secure: why we use isolated VMs

#99

Post by Veserv »

tptacek wrote: post #91
Veserv wrote: post #87

Earlier quoted context omitted.

VT-x. You should get the name of the technology right before defending it. VT-d is the I/O virtualization technology. When did it become customary to defend people making claims of security instead of laughing in their face even though history shows them such claims to be a endless clown parade? How about you present the extraordinary evidence needed to support the extraordinary claim that there are no vulnerabilitie…

Not an especially impressive flex, but I'm not above trying to dunk on people for misspelling things either, so I'm not going to high-horse you about it (obviously i am). The history of KVM and hardware virtualization is not an endless clown parade. Find a vulnerability researcher to talk to about OpenBSD sometime, though. https://isopenbsdsecu.re/

OpenBSD is not secure by any measure. That Theo happens to be right about the endless clown parade is independent of his ability to develop a secure operating system.

I mean, jeez, even Joanna Rutkowska acknowledges the foundations are iffy enough to only justify claiming “reasonably secure” for Qubes OS.

You are making a extraordinary claim of security which stands diametrically opposed to the consensus that things are easily hacked. You need to present extraordinary evidence to support such a claim. You can see my other reply for what I would consider minimal criteria for evidence.

Re: Unfashionably secure: why we use isolated VMs

#100

Post by stacktrust »

compsciphd wrote: post #83
the_duke wrote: post #67

Earlier quoted context omitted.

GVisor basically works by intercepting all Linux syscalls, and emulating a good chunk of the Linux kernel in userspace code. In theory this allows lowering the overhead per VM, and more fine-grained introspection and rate limiting / balancing across VMs, because not every VM needs to run it's own kernel that only interacts with the environment through hardware interfaces. Interaction happens through the Linux syscall…

I actually wonder how much "overhead" a VM actually has. i.e. a linux kernel that doesn't do anything (say perhaps just boots to an init that mounts proc and every n seconds read in/prints out /proc/meminfo) how much memory would the kernel actually be using? So if processes in gvisor map to processes on the underlying kernel, I'd agree it gives one a better ability to introspect (at least in an easy manner). It give…

https://github.com/Wenzel/pyvmidbg

  LibVMI-based debug server, implemented in Python. Building a guest aware, stealth and agentless full-system debugger.. GDB stub allows you to debug a remote process running in a VM with your favorite GDB frontend. By leveraging virtual machine introspection, the stub remains stealth and requires no modification of the guest.
more: https://github.com/topics/virtual-machine-introspection
Post reply on HN