Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The What, Why and How of Containers

annwan.me

Post reply on HN
Print view View original

61–70 of 132 posts

Re: The What, Why and How of Containers

#61

Post by adamgordonbell »

tutfbhuf wrote: post #48
adamgordonbell wrote: post #2

If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…

One issue with static linking is that your dependencies will likely have critical CVEs over time. If you keep all your libraries separate on the filesystem, you can just do a "apt update; apt upgrade", and you will have all the latest patches. This will patch security issues in e.g. libssl or libc for all your applications that are dynamically linked against this shared libraries, which can be quite a few. In static…

That all makes sense. But when those 100 binaries end up as 100 OCI images, and then to patch them you need to update those 100 OCI images to have the new version, it does seem like we've gone in a circle a bit.

I mean, there are some advantages, if they all share the same base layer, maybe they share those libs at least on disk via a shared layer. But practically, though you are maybe not back where you started, you are at a place that seems to share some similarities.

Re: The What, Why and How of Containers

#62

Post by nurple »

cwillu wrote: post #36

Computing is an endless cycle of inventing ways to isolate code in a private machine, followed by inventing ways to make it easier for those machines to interoperate.

This is a really interesting way to think about the progression.

As a timeline I like to plot the ratio of users to isolated compute. We've moved along points like users per building, users per room, user per computer, computers per user, kernels per user, processes per user.

Containers enabled the latest shift.

Re: The What, Why and How of Containers

#63

Post by morcus »

sltkr wrote: post #30
adamgordonbell wrote: post #19

Earlier quoted context omitted.

> If you want to reuse the shared parts of binaries But aren't we, when we use OCI images as a packaging mechanism, using containers to essentially throw away that sharing and arrive at a complicated version of static linking, where everything dynamically linked is shipped with the program? Same goes for arguments about ease of patching things. When the software's package is actually an image, you are patching each i…

No, because you can share common libraries across containers by putting them in a separate layer. For example, if you have a complex service that consists of multiple binaries all written in C++ using boost, then for each binary you can create a container that contains a layer of a base OS (shared), C++ libraries (shared), boost libraries (shared), application binary (unique). All the services can now share their com…

> All the services can now share their common libraries, both on disk and in memory, which reduces I/O and memory use.

Wow, how is this possible using layers? How does docker handle it if I subsequently modify one of the files of my layer in my container?

Re: The What, Why and How of Containers

#64

Post by zer00eyz »

bayindirh wrote: post #50
zer00eyz wrote: post #35

Earlier quoted context omitted.

>>> To me, all that points at containers being in some way a solution to Dynamic linking. And maybe an over the top solution. ... Should we be doing more static linking? Not even depending on libc? What are the challenges with that? >> but it's a ruby application and I need to install the whole stack of things, plus the gems. Instead I containerize it, and keep my system clean. There was a bit of drama recently with…

> Bottles devs saying "we only want the appimage as a distribution method". Good for them having wishes, yet free software doesn't work that way in most cases. :) > I see containers as a means of distributing software That's OK. Docker is a tool, and tools can be held and used in many ways.

>>> Good for them having wishes, yet free software doesn't work that way in most cases. :)

Historically I would agree with you.

But bottles, and the attitude of some upstream projects is one of "modern", "faster", "support the latest version only" ... Software devs who live in user space want a more "App Store" like delivery mechanism where they have control. The kernel is inclined to never break backwards compatibility. Distros are sort of stuck in the middle...

>> I see containers ... > That's OK. Docker is a tool,

One of these is a subset of the other... (depending on your perspective). Dockerdesktop running a linux image on a windows box is most certainly a tool... that same container being ported to lcx in production makes it a package manager. Who is a subset of whom is something we could debate but these things can be both.

Re: The What, Why and How of Containers

#65

Post by bayindirh »

zer00eyz wrote: post #64
bayindirh wrote: post #50

Earlier quoted context omitted.

> Bottles devs saying "we only want the appimage as a distribution method". Good for them having wishes, yet free software doesn't work that way in most cases. :) > I see containers as a means of distributing software That's OK. Docker is a tool, and tools can be held and used in many ways.

>>> Good for them having wishes, yet free software doesn't work that way in most cases. :) Historically I would agree with you. But bottles, and the attitude of some upstream projects is one of "modern", "faster", "support the latest version only" ... Software devs who live in user space want a more "App Store" like delivery mechanism where they have control. The kernel is inclined to never break backwards compatibil…

> Software devs who live in user space want a more "App Store" like delivery mechanism where they have control.

They can have control. They can say that we have flatpaks and app images we publish and support, and they only support the latest version (or the same minor version, whatever), and can say that packages got from distros may not be the latest.

There's no need for a yelling match, IMHO.

Distros also can do whatever they want. Like rclone. You can get the packages from rclone.org or from your distros. There's no yelling match, but trade-offs.

> ...that same container being ported to lcx in production makes it a package manager.

I don't think so. I have containers which work like binaries (in the form of "./binary infile outfile" fashion) and exit after processing what I give them. For me that container is a utility program as a whole. Same for the imap-backup example.

When you think services which are always on, docker might be a package manager, but I pack my own containers for example, so it's more like compiling for me.

So, what docker or containers is depends on your perspective, or like a chameleon which changes its color according to the landscape it's in.

So, it's a tool in the end. Package managers are tools, too.

Re: The What, Why and How of Containers

#66

Post by sloowm »

zer00eyz wrote: post #35
bayindirh wrote: post #4

Earlier quoted context omitted.

Disclaimer: I'm not a strong containerization proponent. The good part of containers is you isolating the thing you're running. I'm very against resource waste, but if I can spend 90MB on a container image instead of installing a complete software stack to run a task which is executed weekly and runs for 10 minutes, I'd prefer that. Plus, I can create a virtual network and storage stack around the container(s) if I n…

>>> To me, all that points at containers being in some way a solution to Dynamic linking. And maybe an over the top solution. ... Should we be doing more static linking? Not even depending on libc? What are the challenges with that? >> but it's a ruby application and I need to install the whole stack of things, plus the gems. Instead I containerize it, and keep my system clean. There was a bit of drama recently with…

I think it was Flatpak that Bottles prefers.

The thing with Bottles is that it also relies on a very specific setup of all underlying software. So in that case it's more about complexity that leads to the need of containerization than keeping systems clean or containerizing for sandboxing.

Re: The What, Why and How of Containers

#67

Post by nurple »

adamgordonbell wrote: post #2

If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…

Containers are a solution to dependency management for sure. In the world of the FHS, the dynamic linker is meant to solve a number of problems, like space saving and security updates, at the OS level by discovering dynamic deps installed in special library paths. One thing I've never liked about FHS is how everything is organized by kind.

The interesting thing about how Nix approaches the problem is to replace the concept of FHS almost entirely (only a couple binaries are linked to /) by hijacking PATH, and the linker configs like you mentioned. The biggest difference being that the whole version-pinned dep tree is encoded in a nix package(and in the linker config of the binaries it produces) rather than just the package itself.

At some level you could say there is no "dynamic" runtime linking in nix, i.e the linker uses partially specified deps in a discovery phase, all of the link bindings happen at build time.

The FHS did attempt to solve the issue of multi-version dependencies with an interesting name and symlink setup, but they are usually still bound by fairly loose version constraints (like major version). Containers are a lot more like nix in this way, where deps are "resolved" at build time by the distro's package manager by virtue of controlling the process' filesystem.

This is one major issue with the reproducibility of container builds, the distro package managers are not deterministic, you could run a build back-to-back and get different deps depending on your timing(yes, even between test and build CI steps).

Re: The What, Why and How of Containers

#68

Post by disconnect3d »

It's a nice blog post but it still misses a few important building blocks without which it would be trivial to escape a container running as root.

Apart from chroot, cgroups and namespaces, the containers are also build upon:

1) linux capabilities - that split the privileges of a root user into "capabilities" which allows limiting the actions a root user can do (see `man 7 capabilities`, `cat /proc/self/status | grep Cap` or `capsh --decode=a80425fb`)

2) seccomp - which is used to filter syscalls and their arguments that a process can execute. (fwiw Docker renders its seccomp policy based on the capabilities requested by the container)

3) AppArmor (or SELinux, though AppArmor is the default) - a LSM (Linux Security Module) used to limit access to certain paths on the system and syscalls

4) masked paths - container engines bind mounts certain sensitive paths so they can't be read or written to (like /proc/sysrq-trigger, /proc/irq, /proc/kcore etc.)

5) NoNewPrivs flag - while not enabled by default (e.g., in Docker) this prevents the user from gaining more privileges (e.g., suid binaries won't change the uid)

If anyone is interested in reading more about those topics and security of containers, you may want to read a blog post [0] where I dissected a privileged docker escape technique (note: with --privileged, you could just mount the disk device and read/write to it) and slides from a talk [1] I have given which details the Docker container building blocks and shows how we can investigate them etc.

[0] https://blog.trailofbits.com/2019/07/19/understanding-docker...

[1] https://docs.google.com/presentation/d/1tCqmGSOJJzi6ZK7TNhbz...

Re: The What, Why and How of Containers

#69

Post by cjk2 »

cwillu wrote: post #36

Computing is an endless cycle of inventing ways to isolate code in a private machine, followed by inventing ways to make it easier for those machines to interoperate.

Don't forget an endless cycle of inventing ways to make debugging and problem solving harder by adding isolation boundaries and complexity :)

Re: The What, Why and How of Containers

#70

Post by palata »

zer00eyz wrote: post #59
palata wrote: post #18

Earlier quoted context omitted.

They do more than that. For instance, they can be swapped easily. Think for instance a security library being updated by the distro security team. It also makes it easier to depend on an LGPL library, nicely allowing the users to modify the LGPL library without having to recompile your program.

>>> Think for instance a security library being updated by the distro security team. And when your distro doesn't update? When the distro is slow to change? Flatpak, Appimage, Snaps (why?)... upstream devs are bypassing distro maintainers in a lot of cases.

I am not saying that they solve all problems in the world. I am just saying that they have merits.

In your particular example, if you are not happy with your distro, you are free to find another one. Some distros are extremely fast to change.

> upstream devs are bypassing distro maintainers in a lot of cases

IMO, for open source projects, upstream devs should not distribute their library. They should let package maintainers do it for their distro.

Post reply on HN