Transitioning from Docker to Podman

https://github.com/rancher/k3c https://github.com/genuinetools/img

(And again sorry for my rather harsh words - I have a lot of respect for all the hard work everyone at Red Hat is doing. I can only imagine that it's much harder to make technical decisions and build tools like Podman than it looks like. Thank you for not taking my rant as an offence! It's just my brutally raw user experience).

I don't think he's FUDding about systemd. I may have misinterpreted, but I think he is saying that his application (postfix in the example he gave) requires systemd as a dependency (for logging in this example). I don't think he is saying that podman, et al require systemd.

I would respectfully offer as well that from your perspective there's no fragmentation (and I think you make a great point about that), but from the perspective of a typical developer or sys-admin, there is absolutely fragmentation. A lot of devs/sysadmins don't only work with Red Hat, they have other linuxes to deal with as well. To them, it used to be that `docker` and everything about it was the standard. Whether you wanted to develop on Fedora or Ubuntu, it didn't matter, you just used `docker`. When you went to prod, it was just docker. Everyone on the internet writing blog posts and tutorials were using `docker`. There really wasn't anything else that was popular (although I was personally rooting for rkt and systemd-nspawn but those didn't really take off ;-)

Nowadays, there is "fragmentation" because depending on you production platform, your OS of choice, architect whims, or whatever, at any given time you may be dealing with docker, or podman, or CRI-O etc.

That said, my personal perspective is closer to yours. I agree with your arguments, and I'm a big fan of Podman et al and I don't see it as fragmentation. When I put on the perspective of an average dev/sysadmin (which I work closely with every day), some of the terms (like "fragmentation" change meaning a bit).

The point that I wanted to make about systemd is that in practice I'm again and again running into systemd vs. containers problems. Systemd isn't really intended to run in containers. There are workarounds (such as sharing special files from the host with the container or using Podman that has implemented similar workarounds for it). The reason is that on modern "fat" Linux distributions (like RHEL/CentOS or Ubuntu) more and more things depend on systemd. Postfix is just a simple example. Because of this clash the original promise of containers (package any user mode stuff into a container and run in anywhere) often feels broken. This isn't Podmans fault (Podman is doing what it can to ease the pain, but it only works when you use Podman). A container image that contains systemd needs special treating (not possible on any host or platform that can run containers) and isn't really a portable container anymore.

These kind of issues tend to let me avoid using CentOS/UBI as container base image and instead go with i.e. Alpine that doesn't have the things I like about EL (LTS, etc) but also never has any mixups with systemd. (It's a bit sad having to go that road when your workplace actually pays for EL support).

I wouldn't have any suggestion how to easily fix this though. Systemd will never fully run in a container I suppose (it's too much linked to non userspace) nor will all container tools and platforms support workarounds for it.

Aside from systemd here's another example where I think Podman UX could be better. Running containers in containers is an important usecase, even if it's insecure. Docker has provided workarounds for this early on. Yes, horrible in terms of security. But good enough for CI/CD runners. Either you can pass the Docker sock file from host to guest. Or in a --privileged container you can run a Docker engine and connect to it from another Docker container. It's well documented and there is a ready "docker:dind" (docker in docker) container image.

My #1 hope with Podman rootless containers was that I could just run "podman run ..." inside a podman container. But it turns out it's (sill) not that easy, even with "--storage-driver=vfs".

What's the recommended approach for running containers in containers with a version of Podman that is available today in the stable version of RHEL/CentOS? I lack to find docs about it. I assume you can do some unprivileged foo with Podman, too ... but how?

(I know latest Podman2 is providing a largely Docker compatible sock file, but I understand its early? Don't misunderstand me, this is great, will use it once it lands in RHEL. But until then ... Docker for this?).