Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Barco: Linux Containers from Scratch in C

github.com

Post reply on HN
Print view View original

31–40 of 75 posts

Re: Barco: Linux Containers from Scratch in C

#31

Post by viraptor »

zamalek wrote: post #7

> barco enforces a minimal set of restrictions to run untrusted code, which is not recommended for production use, where a more robust solution should be used. Aren't containers never suitable for running untrusted code? You need AppArmor, bwrap, or similar AFAIK.

Depends what you mean by suitable. If you run the service as a new user, it's more secure than running without a new namespace (you're isolated from other apps) and potentially less secure than running on host (one more layer of indirection for system resource access).

Since in reality most attacks will be against your app itself before the attacker has direct access to syscalls, I see namespaces/containers as extra protection.

Re: Barco: Linux Containers from Scratch in C

#32

Post by creatonez »

zamalek wrote: post #7

> barco enforces a minimal set of restrictions to run untrusted code, which is not recommended for production use, where a more robust solution should be used. Aren't containers never suitable for running untrusted code? You need AppArmor, bwrap, or similar AFAIK.

If built to spec, then the various container technologies in the kernel used together are theoretically secure. It closes all of the holes that we know about, aside from a few trivial things like the container spying on process id numbers on the host, and of course the vast potential to accidentally misconfigure it.

However, all this code is quite complex, and the kernel and the software ecosystem are lacking in having a layered approach to security that goes all the way down to the low-level nitty gritty stuff. For example, kernel memory structures are not robustly protected against the usual memory exploits, and there isn't as strong W^X protection as desired. Windows, in contrast, is able to provide layered security through a variety of approaches, including running the entire operating system in a virtual machine, with the host ensuring integrity of kernel memory. These sorts of layered approaches to security are desirable because there will always be defects in any complex software.

Side note: AppArmor and bwrap are distinct. Bubblewrap is a relatively simple userspace program that makes use of existing kernel containerization features (the same ones that Docker/Podman use), whereas AppArmor and SELinux are security features that are patched into the kernel itself. AppArmor and SELinux have made some progress in adding layered low-level security to the kernel, but it's not particularly impressive. Bubblewrap has done great work in exposing the kernel's existing tech to users, but they are not fundamental improvements to the kernel itself.

Re: Barco: Linux Containers from Scratch in C

#34

Post by viraptor »

creatonez wrote: post #32
zamalek wrote: post #7

> barco enforces a minimal set of restrictions to run untrusted code, which is not recommended for production use, where a more robust solution should be used. Aren't containers never suitable for running untrusted code? You need AppArmor, bwrap, or similar AFAIK.

If built to spec, then the various container technologies in the kernel used together are theoretically secure. It closes all of the holes that we know about , aside from a few trivial things like the container spying on process id numbers on the host, and of course the vast potential to accidentally misconfigure it. However, all this code is quite complex, and the kernel and the software ecosystem are lacking in hav…

> aside from a few trivial things like the container spying on process id numbers on the host

Containers with own PID namespace can't spy on process IDs on the host though? Not sure what you mean here.

> and there isn't as strong W^X protection as desired

What level is desired? Bootup warnings for W^X got merged a while ago. Changes that try to include anything violating it are rejected (see bcachefs).

> Windows, in contrast, is able to provide layered security through a variety of approaches, including running the entire operating system in a virtual machine, with the host ensuring integrity of kernel memory.

What? Xen existed for years, that's not "in contrast". Secureboot and lockdown exists on Linux too. There's also per-service firecracker microvm.

> whereas AppArmor and SELinux are new security features that are patched into the kernel itself

That's very misleading. They're not new - selinux is over 2 decades old. They're also not "patched in" - LSMs have been integrated into Linux for a very long time with multiple implementations available. Selinux had multilabel security created for gov use. It's quite impressive actually.

Re: Barco: Linux Containers from Scratch in C

#35

Post by teleforce »

lucavallin wrote: post #2

barco is a project I worked on to learn more about Linux containers and the Linux kernel, based on other guides on the internet.

Looks like a good project to learn container from scratch.

Just wondering the main reason you're C since most of the container project now seems to be using Go or Rust?

Re: Barco: Linux Containers from Scratch in C

#36

Post by creatonez »

viraptor wrote: post #34
creatonez wrote: post #32

Earlier quoted context omitted.

If built to spec, then the various container technologies in the kernel used together are theoretically secure. It closes all of the holes that we know about , aside from a few trivial things like the container spying on process id numbers on the host, and of course the vast potential to accidentally misconfigure it. However, all this code is quite complex, and the kernel and the software ecosystem are lacking in hav…

> aside from a few trivial things like the container spying on process id numbers on the host Containers with own PID namespace can't spy on process IDs on the host though? Not sure what you mean here. > and there isn't as strong W^X protection as desired What level is desired? Bootup warnings for W^X got merged a while ago. Changes that try to include anything violating it are rejected (see bcachefs). > Windows, in…

> That's very misleading. They're not new - selinux is over 2 decades old.

I misspoke on SELinux and AppArmor being "new". What I was getting at is that they are distinct kernel features, compared to bwrap which is just a user of kernel features already familiar in this discussion. So "new", as in, "additional", e.g. "We turned up some new evidence from the old files".

And yes, SELinux is included as a first-class kernel feature. AppArmor is a bit different because it still has a lot of hurdles before all its features can be upstreamed. However, upstreaming is not the end-all-be-all so it's not necessarily a bad thing that parts of AppArmor are patched in, so I'm not emphasizing this point at all.

> What? Xen existed for years, that's not "in contrast".

Does the Xen or KVM ecosystems provide anything comparable to Windows hypervisor-enforced code integrity? That is, the host is aware of what kernel memory needs to be set to read-only or checked regularly for corruption or irregularities, in a system that is impossible to interfere with without a VM break. (https://learn.microsoft.com/en-us/windows-hardware/design/de... )

Secure VMs are great, VMs that are actually monitoring and enhancing the security of the code running inside are even better.

Re: Barco: Linux Containers from Scratch in C

#37

Post by cyphar »

teleforce wrote: post #35
lucavallin wrote: post #2

barco is a project I worked on to learn more about Linux containers and the Linux kernel, based on other guides on the internet.

Looks like a good project to learn container from scratch. Just wondering the main reason you're C since most of the container project now seems to be using Go or Rust?

As the maintainer of a Go container runtime (runc), and having worked with Rust in various other projects, while they can be better languages for building large projects, they make it harder to understand what exactly your program is doing when writing software like this.

One example that immediately comes to mind from Rust is a bug with O_PATH file descriptors I found a while ago[1], which would've made certain code we use in runc not work. And from Go, here is a bug I just found in their code for handling file descriptors for ForkExec[2] which is causing issues in a runc patch I'm working on. Neither of these issues exist in C programs. Though of course, C programs have their own issues. For better or worse, the Linux kernel APIs are easiest to use from C.

In runc we actually implement the core container setup code in C because Go doesn't allow you to do everything we need for setting up a container (it has gotten better though, in the past it was completely impossible to set up a container properly in pure Go -- now you can set one up but there are still certain configurations that are not possible to implement in pure Go, such as "docker exec"). You also cannot run Go in single-threaded mode, which means that certain kernel APIs (unshare(CLONE_NEWUSER) for instance) simply cannot be used from regular Go code.

[1]: https://github.com/rust-lang/rust/issues/62314 [2]: https://github.com/golang/go/issues/61751

Re: Barco: Linux Containers from Scratch in C

#38

Post by [deleted] »

teleforce wrote: post #35
lucavallin wrote: post #2

barco is a project I worked on to learn more about Linux containers and the Linux kernel, based on other guides on the internet.

Looks like a good project to learn container from scratch. Just wondering the main reason you're C since most of the container project now seems to be using Go or Rust?

[deleted]

Re: Barco: Linux Containers from Scratch in C

#39

Post by viraptor »

creatonez wrote: post #36
viraptor wrote: post #34

Earlier quoted context omitted.

> aside from a few trivial things like the container spying on process id numbers on the host Containers with own PID namespace can't spy on process IDs on the host though? Not sure what you mean here. > and there isn't as strong W^X protection as desired What level is desired? Bootup warnings for W^X got merged a while ago. Changes that try to include anything violating it are rejected (see bcachefs). > Windows, in…

> That's very misleading. They're not new - selinux is over 2 decades old. I misspoke on SELinux and AppArmor being "new". What I was getting at is that they are distinct kernel features, compared to bwrap which is just a user of kernel features already familiar in this discussion. So "new", as in, "additional", e.g. "We turned up some new evidence from the old files". And yes, SELinux is included as a first-class ke…

> Does the Xen or KVM ecosystems provide anything comparable to Windows hypervisor-enforced code integrity?

Xen can do it, KVM had some attempts at memory enforcement patches, but not sure where it ended up. I'm not sure how well utilised it is though from the guest side ootb. (I think poorly)

A lot of the signing/module loading issues are prevented on the guest side though with https://man7.org/linux/man-pages/man7/kernel_lockdown.7.html

So it's... not the same, but the facilities are available. (Unless someone wants to correct me and it's already used)

Re: Barco: Linux Containers from Scratch in C

#40

Post by serf »

teleforce wrote: post #35
lucavallin wrote: post #2

barco is a project I worked on to learn more about Linux containers and the Linux kernel, based on other guides on the internet.

Looks like a good project to learn container from scratch. Just wondering the main reason you're C since most of the container project now seems to be using Go or Rust?

I can't answer for the developer, but the answer to that with most small one-person-show projects is familiarity/comfort/ability.

the head-space that adopting a new language for a specific project takes is immense compared to tackling it in a familiar language that you know you're already able in; there is rarely a benefit to doing so outside of team environments where a certain level of on-boarding is expected, or because you have a really niche language requirement/feature that your project is begging for.

Post reply on HN