Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZeroVM: lightweight containers based on Google Native Client

zerovm.org

Post reply on HN
Print view View original

51–60 of 79 posts

Re: ZeroVM: lightweight containers based on Google Native Client

#51

Post by camuel »

ww520 wrote: post #43
eklitzke wrote: post #39

Earlier quoted context omitted.

I think in practice there have been a few historical problems with this approach. A major one is that while in theory it's possible to secure a process to a chroot jail (either the Linux or the BSD variety) in practice it's hard to defend against arbitrary, possibly malicious programs in that jail. Kernel exploits that rely on local access are uncommon but not incredibly rare. You could easily have a hosting service…

Thanks for the responses; however, most of the attacks to OS process can happen in ZeroVM. - Kernel exploits can happen, so can exploits in ZeroVM. - DOS attack on OS, so can DOS attack on ZeroVM. I don't see any quota system or resource management in ZeroVM to mitigate it. At least the OS have better tools to deal with it - priority, scheduler, resource manager, memory partition, etc. - Same thing for the customer w…

ZeroVM is not a lightweight container so all the above doesn't hold. ZeroVM doesn't use any "syscall firewalling" techniques. ZeroVM efficiently emulates new hardware platform just as XEN/KVM.

Now let me address your concerns one by one:

1. (Host) Kernel exploits cannot happen as no syscalls to host OS are allowed. ZeroVM app doesn't even have such a concept as host OS.

2. DoS attacks impossible on host OS as there are no access to host OS. The interface between ZeroVM application and ZeroVM itself is specifically designed to be impossible to DoS attack. It consists now of 4 functions. Setup and exit are callable only once during lifetime of app. Message queue read/write could be repeatedly called but the it intentionally designed to be synchronous, so throttling mechanism could be transparently implemented.

3. We intentionally haven't hacked NaCl validator as it the only component in the system that guaranties security. Google established 5-6 digit monetary prizes for any exploits in Chrome/NaCl and heavily invest in security. For many customers it is enough.

4. No one asserted that ZeroVM is mature and right now it is not! So this piece of advice is correct. If you need security now for production usage - KVM/XEN is the only way to go.

5. OS security... So much work was done to secure OS from outside... not from inside. I would appreciate description how it is easy to lock-down OS process in multi-tenant sense. Also when you start "syscall firewalling" and draconian restriction it would be very hard to program such as system. Just think for a moment. You take the whole syscall list, for every syscall you decide on restrictions (I haven't found any document on web on that). Now how you work with such API? How you enforce yourself before you issue a syscall, what to do when third-party code is causing violations? Syscall API is not built for such draconian capping....

Re: ZeroVM: lightweight containers based on Google Native Client

#52

Post by justincormack »

ww520 wrote: post #46
prodigal_erik wrote: post #45

Earlier quoted context omitted.

Has OS isolation ever been implemented successfully? I think I've seen local root exploits reported for every OS I've heard of.

And what are the evidence of VM based isolation have more security?

It is a layered approach, first you wold have to find an exploit in the "VM" (which is a sandbox really) thn exploit the underlying OS. The VM has a much smaller attack surface as you have less you can do, so it is easier to audit. NaCl, which is used here has had minor flaws http://arstechnica.com/open-source/news/2009/07/google-nacl-... but nothing like straight kernel. Sure there are other approaches eg see http://sandboxing.org/ eg to use selinux to constrain processes, but none are easy. There is some more recent work on more directly limiting syscalls to processes which is another approach, so the OS provides an isolation service.

Re: ZeroVM: lightweight containers based on Google Native Client

#53

Post by justincormack »

camuel wrote: post #42
eklitzke wrote: post #39

Earlier quoted context omitted.

I think in practice there have been a few historical problems with this approach. A major one is that while in theory it's possible to secure a process to a chroot jail (either the Linux or the BSD variety) in practice it's hard to defend against arbitrary, possibly malicious programs in that jail. Kernel exploits that rely on local access are uncommon but not incredibly rare. You could easily have a hosting service…

Thanks for that, I feel the same way for process isolation. In theory that should be a solution but in practice doesn't work for historic reasons. Just one comment. ZEROVM IS NOT NACL. It uses NaCl, moreover we explicitly refrained to touch validator in order to remain under its proven security blanket (Google established hefty monetary prizes for each found exploit). However, except of validator it is heavely refact…

ARM support? Excellent, was going to look at adding this.

Re: ZeroVM: lightweight containers based on Google Native Client

#55

Post by camuel »

ww520 wrote: post #46
prodigal_erik wrote: post #45

Earlier quoted context omitted.

Has OS isolation ever been implemented successfully? I think I've seen local root exploits reported for every OS I've heard of.

And what are the evidence of VM based isolation have more security?

There are two techniques for resources isolation:

1. Filtering all resources accesses, letting some pass and others denied.

2. Enforcing different abstraction and then unwanted resources accesses become impossible as they are not even addressable.

Filtering is by definition less secure. As filtering get more complicated there would be false negatives and false positives. Both are harmful.

Enforcing different abstraction is usually less efficient as there is a need to simulate hardware devices. However, some devices have hardware support for virtualization as with Intel CPUs and MR-IOV devices and then enforcing abstraction is free.

LXC/OpenVZ uses mostly #1

XEN/KVM uses mostly #2

Re: ZeroVM: lightweight containers based on Google Native Client

#56

Post by camuel »

ww520 wrote: post #44
camuel wrote: post #41

Earlier quoted context omitted.

1. ZeroVM abstraction is C including full support for native inline assembly/intristics. Make your own conclusions about portability with regards to platform. OS portability could/will be provided and it is piece-of-cake (for geeky overoptimism of course) 2. Don't want to go into JIT / C flamewar... but let's take a sorting example. In samples directory you can find my hand-coded assembly code (thank you Takeshi Yama…

I don't understand why multi-tenant is problem. Multi-user Unix (or others) have supported allowing multiple untrusted users/programs to run for a long time. Edit: I don't mean to be a downer on your project. It's a cool project, but I think you would prefer constructive criticism than me just saying "awesome, carry on."

Constructive criticism is always welcome.

The answer is that multi-tenant is different from multi-user on so many aspects.

Multi-user: 1. Usually closed institution or even single group within single institution where all know each other. 2. Usually a lot of shared data and application. 3. The risk of malicious activity is low. 4. The consequences of malicious activity are not severe/fatal. 5. Auditing followed by identification and penal actions are an effective strategy to stop malicious activity.

Multi-tenant: 1. Usually open to anyone without "background checks". 2. Single instance of malicious activity is severe, repeated malicious activity is fatal. 3. Auditing, identifying the intruder and suing him in court is not an effective strategy to stop malicious activities.

Re: ZeroVM: lightweight containers based on Google Native Client

#57

Post by camuel »

shykes wrote: post #40
camuel wrote: post #32

Earlier quoted context omitted.

exactly! Further from being a different abstraction, container technologies (at-least in their current implementations of 'chroot on nukes') are not completely sealed or 'secure'. OpenVZ seems to be the most secured one over-there, requires kernel-patching and still... close but not 100% airtight. That is one of the reasons that many lightweight containers are used only as secondary sandbox (like Heroku) and not allo…

dotCloud ( http://dotcloud.com ) supports arbitrary code execution inside LXC containers (pre-2010 versions used OpenVZ, and very early versions were built on V-server). The main limitation is that the process runs under an unprivileged uid under a kernel managed and deployed by dotCloud. I agree with the assessment that containers are not "completely secure" - I would not trust it to contain a root-privileged proces…

What if I DoS attack some syscall? Or create zillions of files with 1 byte size driving crazy file-system or anything else.

Kernel is such vast area vulnerable for an attack that it is scary even to think about securing all of it and not leaving a single weak point. Moreover, you will screw your syscall API to the point that it will become unusable. At bare least we need standard for the syscall capping and etc... so programmer will know what to expect.

And thanks for the link, will check them and what solution they use and whether they are happy with it.

Re: ZeroVM: lightweight containers based on Google Native Client

#58

Post by karterk »

what is less known is that when deployed at cloud, Hadoop cannot access that enormous dataset locally due to security restrictions and therefore is screamingly inefficient compared to on-premise Hadoop deployment.[1]

Can someone explain what that means?

[1]: http://zerovm.org/killer-apps/

Re: ZeroVM: lightweight containers based on Google Native Client

#59

Post by [deleted] »

karterk wrote: post #58

what is less known is that when deployed at cloud, Hadoop cannot access that enormous dataset locally due to security restrictions and therefore is screamingly inefficient compared to on-premise Hadoop deployment. [1] Can someone explain what that means? [1]: http://zerovm.org/killer-apps/

[deleted]

Re: ZeroVM: lightweight containers based on Google Native Client

#60

Post by camuel »

karterk wrote: post #58

what is less known is that when deployed at cloud, Hadoop cannot access that enormous dataset locally due to security restrictions and therefore is screamingly inefficient compared to on-premise Hadoop deployment. [1] Can someone explain what that means? [1]: http://zerovm.org/killer-apps/

If you use EMR or just roll your own Hadoop in EC2 then:

1. Hadoop runs on EC2 2. Data is stored on S3 3. Intermediary results stored in EC2 4. Hadoop loads the data from S3 to EC2 5. EC2S3 bandwidth is not that fast or efficient (S3 proxy, network contention, TCP/IP processing)

Hypothetical MapReduce/ZeroVM/Swift scenario:

1. Data is stored on S3/Swift 2. Map and Reduce functions are run inside S3/Swift secured by ZeroVM in majority of cases accessing data locally without networking/proxies getting in the way. 3. Intermediate and final results are also stored within S3/Swift. 4. Local data access is efficient, fast and predictable 5. Local networking within S3/Swift is more efficient, fast and predictable than S3EC2 / SwiftNova

Accelerated Hadoop scenario:

Exactly as in #1, just Hadoop makes "predicate pushdown optimization" into S3/Swift secured by ZeroVM.

Regarding 'due to security restrictions' I meant that cloud vendor would not let you run your own code in S3 or CloudFiles. Why? Because you could mess up other people data and storage system itself. Why not run in VM inside S3? well I guess it would be impractical due to long provisioning time of conventional VM.

Post reply on HN