Unfortunately this skips over the history of microkernels, which solve the same problems in a much more elegant way than containers.
The What, Why and How of Containers
11–20 of 132 posts
Re: The What, Why and How of Containers
#12Great high level explanation.
Note: if you look into the details of how setting up namespaces and cgroups works you'll run away in horror. The APIs are very iteratively evolved piecework, not really a coherent(ly designed) abstraction.
Re: The What, Why and How of Containers
#13If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…
Funny - it looks like you’re being downvoted for asking what I think is a very natural question. It’s one I’ve asked before; have we just created a more elaborate statically-linked executable via containerization? In the end, Docker/OCI seems like the universal Linux package manager. I’m sure I don’t have the full picture since I’m far more ops than dev though.
I don't think so. Instead we created an artifact which can live inside an OS, but cannot see and touch to the rest of the OS.
CGroups is a deceptively powerful mechanism. You can isolate a process resource wise (X cores, Y amount of memory, Z amount of swap), network wise (a different virtual network adapter with its own IP, bandwidth limits, etc.) and FS wise (running in its own filesystem with devices it can see).
It can keep your system tidy by encapsulating elaborate stacks which makes system management painful, allows deterministic operation and image generation if you tag everything with version.
Downside is you can do bad things with it like terminating HTTPS with a gateway container and talk HTTP among your backend instead of configuring tools, or writing shoddy software, and getting away with it because it works, or gets automatically restarted when it crashes 6 times a day.
I don't run every service as a container, because some services suffocate when they are in a container, but for short running things which needs system-wide changes to function, or test-driving small services before fully committing, it's a good tool to have.
However, it's abused with no end, and their popular use leaves a bad taste in your mouth.
Re: The What, Why and How of Containers
#14Unfortunately this skips over the history of microkernels, which solve the same problems in a much more elegant way than containers.
Can you elaborate?
Re: The What, Why and How of Containers
#15If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…
dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.
As a bonus, shared libraries are deduplicated in RAM, too.
Plus you don't need to recompile the universe because of a patch.
Re: The What, Why and How of Containers
#16Earlier quoted context omitted.
Disclaimer: I'm not a strong containerization proponent. The good part of containers is you isolating the thing you're running. I'm very against resource waste, but if I can spend 90MB on a container image instead of installing a complete software stack to run a task which is executed weekly and runs for 10 minutes, I'd prefer that. Plus, I can create a virtual network and storage stack around the container(s) if I n…
> many foreign processes not seeing each-other on the same OS For sure, I was thinking of the packaging nature of containers, not the 'security' nature of containers. The pivotroot part. Though I guess being able to have namespaces does make packaging clearer in certain cases. For a horrible analogy: With actual shipping containers, we don't have each shipping container be a stripped down model of a ship, so that the…
The packaging is generally a side-effect of isolation in my experience. I never chrooted a software because I needed a different library stack for it to run, but to isolate it from rest of the system in one way or another (for security/access reasons).
Docker just made the interface more practical, and built the ecosystem around it. lxcontainers, apptainer and podman has improved upon the idea in different ways to cater different use cases.
So for me containers were never simplified ships to begin with. This different perspectives happen probably because people look from different perspectives at initial contact, which is normal.
Re: The What, Why and How of Containers
#17If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…
dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.
They are complementary. If you want to reuse the shared parts of binaries, then you need a way to separate the binary image into parts that are specific to the application and libraries that can be reused across binary images, plus some metadata how to reconstruct the image from its parts. That's what exactly what dynamic libraries provide.
In general, it's much easier to link a binary with its libraries than go in to the opposite direction (extracting common code from static linked binaries), because once you statically link a binary the library code will vary slightly due to differences in memory addresses, compiler optimizations, unused code that has been omitted, different input library versions, etc.
Even if you were, in theory, able to write a complex filesystem driver that is able to extract the common parts of statically linked libraries so they can be deduplicated, then to reconstruct the original binary in memory, you'd have to perform something similar to dynamic linking, except now in the kernel, which really isn't an improvement.
Re: The What, Why and How of Containers
#18If you use chroot to run something, it's interesting how the dynamic libs you need to get in place grows until you are mirroring a whole linux in a subtree. It gives you a sense for how you end up with containers. One thing that is wild to me is how nix solves this problem, of things needing to be linked together. It doesn't solve it with containers, but by rewriting the location of the links in the executable to be…
dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.
They do more than that. For instance, they can be swapped easily. Think for instance a security library being updated by the distro security team. It also makes it easier to depend on an LGPL library, nicely allowing the users to modify the LGPL library without having to recompile your program.
Re: The What, Why and How of Containers
#19Earlier quoted context omitted.
dynamic linking to me seems like solving a problem that filesystems should. deduplicating data.
They are complementary. If you want to reuse the shared parts of binaries, then you need a way to separate the binary image into parts that are specific to the application and libraries that can be reused across binary images, plus some metadata how to reconstruct the image from its parts. That's what exactly what dynamic libraries provide. In general, it's much easier to link a binary with its libraries than go in t…
> If you want to reuse the shared parts of binaries
But aren't we, when we use OCI images as a packaging mechanism, using containers to essentially throw away that sharing and arrive at a complicated version of static linking, where everything dynamically linked is shipped with the program?
Same goes for arguments about ease of patching things. When the software's package is actually an image, you are patching each image individually that is running on the system.