Live data from Hacker News

Abusing Privileged and Unprivileged Linux Containers

nccgroup.trust

21–30 of 54 posts

Re: Abusing Privileged and Unprivileged Linux Containers

#23
post #6

Earlier quoted context omitted.

The original design wasn't intended to provide that kind of isolation, and the primitives that are exposed are retrofit; every new containerization design needs an audit that captures the entire exposed functionality of the Linux kernel. You can just skim this paper to see the problems: non-namespaced identifiers leak in procfs, UID "slides" expose containers to each others resource limits, there are non-namespaced n…

That's interesting...it was my impression that some of the kernel features were added specifically as a result of the kernel patches that were originally part of the OpenVZ project. Once the kernel adopted official primatives the original OpenVZ patches were deprecated. It was also at this time that LXC started with some of the same developers from the OpenVZ project. I could be wrong...but that path dependency seems…

I'm not evaluating the container features in isolation. Considered by themselves, they might be perfectly coherent. The problem is that every feature of the kernel with a namespace of any sort needs to be aware of those container features, and namespaces leak into each other unexpectedly, because most of them are very old and were implemented long before anyone considered containerization.

Re: Abusing Privileged and Unprivileged Linux Containers

#24
post #10

Earlier quoted context omitted.

FreeBSD jails, I would think.

Unless I've missed something (and I may have!), FreeBSD's jails have a very respectable security track record. Really, really want to make use of them. I can't give up Debian's package system, though, so I'm left hoping that kFreeBSD will amount to something someday and I use Xen or KVM in the meantime... :-(

> I can't give up Debian's package system, though

Why not? What would you miss from it?

Re: Abusing Privileged and Unprivileged Linux Containers

#25
post #14

I think what Intel are doing with Clear Containers is really interesting. They are encapsulating containers inside VMs, avoiding the security problems of containers. To do this efficiently they've had to make a bunch of changes on the VM side so the overhead is much smaller than an ordinary VM (of the order of 150ms and 20MB of RAM). I've also been looking at this and am hoping to give a talk about it at the KVM Foru…

150ms for which operation?

150ms to boot the VM that is running the container processes with the stripped down Linux Kernel, lkvm, and DAX. This is about what we observe in the rkt Clear Containers "stage1": https://coreos.com/blog/rkt-0.8-with-new-vm-support/

One note: you can run multiple "container processes", like redis and a redis dashboard, inside of these Clear Container VMs. This means in the case of Kubernetes we will only incur the cost of the startup time and Kernel/init overhead once per pod instead of once per process.

Re: Abusing Privileged and Unprivileged Linux Containers

#26
post #12
post #10

Earlier quoted context omitted.

FreeBSD jails, I would think.

But do enterprise companies use FreeBSD jails regularly? AFAIK they're basically used as toys by developers.

Jails are used on the Playstation 4 and with some 36 Million PS4s sold so far that's a huge use of jails in production. Here's a quote from an article talking about it,

"We can prove the existence of FreeBSD jails being actively used in the PS4's kernel through the auditon system call being impossible to execute within a jailed environment"

This quote is from: https://cturt.github.io/ps4.html

Re: Abusing Privileged and Unprivileged Linux Containers

#27
post #25

Earlier quoted context omitted.

150ms for which operation?

150ms to boot the VM that is running the container processes with the stripped down Linux Kernel, lkvm, and DAX. This is about what we observe in the rkt Clear Containers "stage1": https://coreos.com/blog/rkt-0.8-with-new-vm-support/ One note: you can run multiple "container processes", like redis and a redis dashboard, inside of these Clear Container VMs. This means in the case of Kubernetes we will only incur the c…

This is going to sound amusing at best, but would you clarify what it means to boot?

To justify the question a bit: booting traditionally meant physically turning a system on. The boot time included BIOS initialization, a concept now blurred by the advent of virtualization.

150ms is such an absurdly short amount of time that I'm left wondering what booting is in this context.

Re: Abusing Privileged and Unprivileged Linux Containers

#28
post #9
post #4

Containerization in Linux is fugly . There is no core concept of containers in the kernel, you just have a set of loosely integrated namespaces abused by the likes of lx[cd] and docker.

Fugly? Compared to what alternatives? The offerings from Microsoft are even fuglier .

In illumos, a descendant of (Open)Solaris, we have a first class container primitive called "zones". In SmartOS, the Joyent-backed distribution of illumos, we also have support for running an entire Linux userland (e.g. Ubuntu or CentOS) in this substrate.

You can have the best of both worlds: a secure container substrate, designed from the ground up as a coherent whole like Jails; and the vast packaging ecosystem provided by Ubuntu.

Re: Abusing Privileged and Unprivileged Linux Containers

#29
post #24

Earlier quoted context omitted.

Unless I've missed something (and I may have!), FreeBSD's jails have a very respectable security track record. Really, really want to make use of them. I can't give up Debian's package system, though, so I'm left hoping that kFreeBSD will amount to something someday and I use Xen or KVM in the meantime... :-(

> I can't give up Debian's package system, though Why not? What would you miss from it?

I run Debian Testing and FreeBSD 10. I haven't found too much from Debian that I can't get in FreeBSD 10. I could even run a Debian/kFreeBSD jail if I really wanted to.

What really does my head in is that a default Debian install can pull down 2 megabytes a second from a server over SFTP, and a default FreeBSD 10 server can only do ~800 kilobytes per second (FreeBSD 9 was worse).

Re: Abusing Privileged and Unprivileged Linux Containers

#30
post #25

Earlier quoted context omitted.

150ms to boot the VM that is running the container processes with the stripped down Linux Kernel, lkvm, and DAX. This is about what we observe in the rkt Clear Containers "stage1": https://coreos.com/blog/rkt-0.8-with-new-vm-support/ One note: you can run multiple "container processes", like redis and a redis dashboard, inside of these Clear Container VMs. This means in the case of Kubernetes we will only incur the c…

This is going to sound amusing at best, but would you clarify what it means to boot? To justify the question a bit: booting traditionally meant physically turning a system on. The boot time included BIOS initialization, a concept now blurred by the advent of virtualization. 150ms is such an absurdly short amount of time that I'm left wondering what booting is in this context.

> This is going to sound amusing at best, but would you mind defining what it means to boot?

I imagine that most of the time would be in mocking some/all of the hardware interfaces to present to the VM, and running your init processes (and all that entails for whatever OS you're running).

Post reply on HN