Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The What, Why and How of Containers

annwan.me

Post reply on HN
Print view View original

21–30 of 132 posts

Re: The What, Why and How of Containers

#21

Post by surajrmal »

brabel wrote: post #8
Already__Taken wrote: post #6

Earlier quoted context omitted.

dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.

That's an interesting way to look at it. If all dynamic libs lived on a read-only location, then the file system could actually only store the libs in one place and the other "copies" would be just symlinks to that... and when the OS loaded such lib, it would automatically know that despite being in different locations, the libs were the same (they're all symlinks to the same place). Is this something that has been a…

You can store the files as if you name the actual location of the file as a hash of its contents and symlink the file to that location, you naturally get deduplication. Fuchsia does this [1]. You still end up wanting to try and coordinate your packages to share as many deps as possible for resource optimization reasons, but you no longer depend on it.

[1]: https://fuchsia.dev/fuchsia-src/concepts/filesystems/blobfs

Re: The What, Why and How of Containers

#22

Post by otabdeveloper4 »

kqr wrote: post #7

I wish I had read this article a decade ago. For many years I have been wondering "why the heck would I use containers when I have chroot, cgroups and namespaces?" Turns out that's exactly what containers are a packaging of! And I only found out about two years ago. Although this article doesn't go into it, the benefits I've found of using containers rather than rolling isolation by hand is that a lot of semi-standar…

> Turns out that's exactly what containers are a packaging of!

Well, no. When people say "containers", they always mean "Docker".

And Docker also comes with a daemon with full root permissions and ridiculous security policies. (Like, for example, forcefully turning off your machine's firewall, #yolo. WTF!)

P.S. I actually run systemd-nspawn in production, but I am probably the only person on earth to do so.

Re: The What, Why and How of Containers

#23

Post by otabdeveloper4 »

Already__Taken wrote: post #6
adamgordonbell wrote: post #2

If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…

dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.

No, mostly it solves the problem of runtime library loading and swapping. E.g., Linux VDSO can only work as a shared library.

Re: The What, Why and How of Containers

#24

Post by bayindirh »

otabdeveloper4 wrote: post #22
kqr wrote: post #7

I wish I had read this article a decade ago. For many years I have been wondering "why the heck would I use containers when I have chroot, cgroups and namespaces?" Turns out that's exactly what containers are a packaging of! And I only found out about two years ago. Although this article doesn't go into it, the benefits I've found of using containers rather than rolling isolation by hand is that a lot of semi-standar…

> Turns out that's exactly what containers are a packaging of! Well, no. When people say "containers", they always mean "Docker". And Docker also comes with a daemon with full root permissions and ridiculous security policies. (Like, for example, forcefully turning off your machine's firewall, #yolo. WTF!) P.S. I actually run systemd-nspawn in production, but I am probably the only person on earth to do so.

Docker doesn't turn off the firewall, but rather hijacks and repurposes it for itself. It's not any nicer, but it's not the same thing in reality.

Re: The What, Why and How of Containers

#25

Post by forgotmyinfo »

Already__Taken wrote: post #6
adamgordonbell wrote: post #2

If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…

dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.

Filesystems solve linking insofar as the libraries themselves are on disk; otherwise, this is squarely the responsibility of the loader.

Re: The What, Why and How of Containers

#26

Post by forgotmyinfo »

adamgordonbell wrote: post #19
sltkr wrote: post #17

Earlier quoted context omitted.

They are complementary. If you want to reuse the shared parts of binaries, then you need a way to separate the binary image into parts that are specific to the application and libraries that can be reused across binary images, plus some metadata how to reconstruct the image from its parts. That's what exactly what dynamic libraries provide. In general, it's much easier to link a binary with its libraries than go in t…

> If you want to reuse the shared parts of binaries But aren't we, when we use OCI images as a packaging mechanism, using containers to essentially throw away that sharing and arrive at a complicated version of static linking, where everything dynamically linked is shipped with the program? Same goes for arguments about ease of patching things. When the software's package is actually an image, you are patching each i…

Someone did once joke that Docker is just static linking for millennials. I chuckle because there's a kernel of truth in there. ;)

Re: The What, Why and How of Containers

#27

Post by codethief »

otabdeveloper4 wrote: post #22
kqr wrote: post #7

I wish I had read this article a decade ago. For many years I have been wondering "why the heck would I use containers when I have chroot, cgroups and namespaces?" Turns out that's exactly what containers are a packaging of! And I only found out about two years ago. Although this article doesn't go into it, the benefits I've found of using containers rather than rolling isolation by hand is that a lot of semi-standar…

> Turns out that's exactly what containers are a packaging of! Well, no. When people say "containers", they always mean "Docker". And Docker also comes with a daemon with full root permissions and ridiculous security policies. (Like, for example, forcefully turning off your machine's firewall, #yolo. WTF!) P.S. I actually run systemd-nspawn in production, but I am probably the only person on earth to do so.

> Well, no. When people say "containers", they always mean "Docker".

Not really / not necessarily. https://github.com/opencontainers/runtime-spec

Re: The What, Why and How of Containers

#28

Post by bayindirh »

adamgordonbell wrote: post #19
sltkr wrote: post #17

Earlier quoted context omitted.

They are complementary. If you want to reuse the shared parts of binaries, then you need a way to separate the binary image into parts that are specific to the application and libraries that can be reused across binary images, plus some metadata how to reconstruct the image from its parts. That's what exactly what dynamic libraries provide. In general, it's much easier to link a binary with its libraries than go in t…

> If you want to reuse the shared parts of binaries But aren't we, when we use OCI images as a packaging mechanism, using containers to essentially throw away that sharing and arrive at a complicated version of static linking, where everything dynamically linked is shipped with the program? Same goes for arguments about ease of patching things. When the software's package is actually an image, you are patching each i…

I think a better analogy is, with containers you maintain more "servers" in the end, and patch and reboot them all.

So there's no free lunch and everything is a trade off. People thinking that containers are no more work than managing servers or services are in the wrong.

Re: The What, Why and How of Containers

#29

Post by Rucadi »

adamgordonbell wrote: post #19
sltkr wrote: post #17

Earlier quoted context omitted.

They are complementary. If you want to reuse the shared parts of binaries, then you need a way to separate the binary image into parts that are specific to the application and libraries that can be reused across binary images, plus some metadata how to reconstruct the image from its parts. That's what exactly what dynamic libraries provide. In general, it's much easier to link a binary with its libraries than go in t…

> If you want to reuse the shared parts of binaries But aren't we, when we use OCI images as a packaging mechanism, using containers to essentially throw away that sharing and arrive at a complicated version of static linking, where everything dynamically linked is shipped with the program? Same goes for arguments about ease of patching things. When the software's package is actually an image, you are patching each i…

OCI is not the problem, you can create reproducible images using NIX that reuse all the packages between different images.

The problem is the way we usually package the apps with Docker-Dockerlike builders

Re: The What, Why and How of Containers

#30

Post by sltkr »

adamgordonbell wrote: post #19
sltkr wrote: post #17

Earlier quoted context omitted.

They are complementary. If you want to reuse the shared parts of binaries, then you need a way to separate the binary image into parts that are specific to the application and libraries that can be reused across binary images, plus some metadata how to reconstruct the image from its parts. That's what exactly what dynamic libraries provide. In general, it's much easier to link a binary with its libraries than go in t…

> If you want to reuse the shared parts of binaries But aren't we, when we use OCI images as a packaging mechanism, using containers to essentially throw away that sharing and arrive at a complicated version of static linking, where everything dynamically linked is shipped with the program? Same goes for arguments about ease of patching things. When the software's package is actually an image, you are patching each i…

No, because you can share common libraries across containers by putting them in a separate layer.

For example, if you have a complex service that consists of multiple binaries all written in C++ using boost, then for each binary you can create a container that contains a layer of a base OS (shared), C++ libraries (shared), boost libraries (shared), application binary (unique).

All the services can now share their common libraries, both on disk and in memory, which reduces I/O and memory use. That's one of the main advantages of containers over virtual machines (VMs): each VM instance has a distinct region of memory that is not shared with others even if they happen to load bit-for-bit identical binaries into memory.

(I know, VM memory deduplication exists to ameliorate this problem, but here my previous comment applies: it's much easier to start from shared components and link them together than extract the shared data after the fact. And typically VMs have lots of nonsharable state that containers do share, like pretty much all writable kernel pages.)

Post reply on HN