Live data from Hacker News

Podman: A Daemonless Container Engine

podman.io

171–180 of 250 posts

Re: Podman: A Daemonless Container Engine

#171
post #74

The ability to start containers as a normal user ("rootless mode") is quite interesting and imo the most significant feature missing from LXD, which requires you to either use sudo or be a part of the "lxc" group (equivalent to having sudo privileges).

You can afair configure LXD to disallow escalating privileges via group membership.

Re: Podman: A Daemonless Container Engine

#172
post #112
post #109

I've been trying to use podman in production and it is not working very well. I'm excited for this technology but it is not ready. - podman layer caching during builds do not work. When we switched back to docker-ce, builds went from 45 minutes to 3 minutes with no changes to our Dockerfile - fuse-overlayfs sits at 100% CPU all day, every day on our production servers - podman loses track of containers. Sometimes the…

Reading through the comments though it looks like I should try making a BTRFS mount for the container storage and that might help the fuse-overlayfs problems though.

It feels weird that a containerisation technology imposes requirements on the underlying storage technology to work well, tho.

Now I understand that btrfs is not really a requirement.

But I've been thinking about using podman to replace docker-in-docker on our fleet of gitlab runners where we build our images, and running btrfs is a deal breaker. We really don't want to add complexity to the mix. Fuse-overlay burning through the whole CPU is not really good looking either. Mh.

I still have to properly dig deep into this, but I'm keeping my hopes high.

Re: Podman: A Daemonless Container Engine

#173
post #109

I've been trying to use podman in production and it is not working very well. I'm excited for this technology but it is not ready. - podman layer caching during builds do not work. When we switched back to docker-ce, builds went from 45 minutes to 3 minutes with no changes to our Dockerfile - fuse-overlayfs sits at 100% CPU all day, every day on our production servers - podman loses track of containers. Sometimes the…

I could very well be wrong but Podman seems to have missed the time-frame of opportunity. It was always a knife fight between Red Hat and Docker with regard to tooling. Red Hat wanting to own the toolchain for containers so they didn't have to deal with, and so they could box out, competitors like (now basically defunct) Docker Enterprise.

I've taken a look at podman from time to time over the years but it seems like it's just never formalized, never been polished and almost always has been sub-par in execution. On this list the builds and container control are things that I've run across. I guess - what's the point? The rootless argument leaned on so heavily is pretty much gone, the quality of Podman hasn't (seemingly) improved and now IBM owns Red Hat (subjective, but a viable concern/consideration given what's recently happened with CentOS).

You're more than safe leveraging Docker and buildkit (when and where needed). Quite honestly, given the relatively poor execution of Red Hat with these tools over the years, I don't see the point. I'm sure there are some niche use cases for Podman/Buildah, but overall it just seems to come up as an agenda more than an exponentially better product at this point. Red Hat could have made things better, instead they just created a distraction and worked against the broader effort in the container ecosystem.

Re: Podman: A Daemonless Container Engine

#174

Earlier quoted context omitted.

Go is actually a really poor choice for the container runtime because much of the container setup cannot be done from multithreaded code[0], so it has to be done in C before the go runtime initializes. I do think rust is a better choice for this layer than C because there are still security risks, but getting rid of Go for this layer is a win. I'm not sure why RH chose to rewrite it in C rather than using rust[1]. [0…

> Go is actually a really poor choice for the container runtime because much of the container setup cannot be done from multithreaded code[0] This was addressed in 2017/2018 [0], it's no longer a poor choice. [0]: https://github.com/golang/go/commit/2595fe7fb6f272f9204ca3ef...

While the particular issue of network namespaces and locking the os thread was fixed, there is still c code that must run before the go runtime starts to work around the issue that you cannot do some of the necessary nsenter calls once you have started additional threads. The c code to make runc work is encapsulated in libcontainer/nsenter[0]

[0]: https://github.com/opencontainers/runc/tree/master/libcontai...

Re: Podman: A Daemonless Container Engine

#175
post #159

Earlier quoted context omitted.

That could be, but if Frazelle did actually walk around with a button saying “I will not merge patches for systemd” I think it's simpler and safer to assume Red Hat decided that relying on Docker was risky given how important it was to their future products and the apparent animus on display from the Docker maintainer. If I carpool with someone and they like to go around and loudly proclaim "I refuse to listen to any…

It’s no secret that there were tensions betwen Docker and Red Hat that culminated around 2016, when Red Hat’s desire to capitalize on the success of containers ran into Docker’s own ambitions to compete with Red Hat. Those tensions were eventually resolved (or at least diminished) by Red Hat shifting gears to Kubernetes as their “next Linux”. The drama you’re talking about is from that period - 2015-16. But “crun” wa…

> It’s just a tremendous waste of energy

> The world’s container infrastructure runs on containerd, runc and docker

I don't consider alternate implementations of widely used software to be wasted evergy almost ever. Was Clang a waste of energy because everyone was using GCC? There are many reasons why multiple implementations may be useful, competition and being able to cater to slightly different common use cases obvious ones.

I'm not sure why any end user would wish for there not to be multiple open source projects looking to satisfy a similar technical need.

> in the end will be problematic for Red Hat

That may be, but I don't really care about whether it's good for Red Hat. I care that it increased user choice and maybe at some point features and capabilities offered to users (whether through podman or pressure on docker).

Re: Podman: A Daemonless Container Engine

#177
post #109

I've been trying to use podman in production and it is not working very well. I'm excited for this technology but it is not ready. - podman layer caching during builds do not work. When we switched back to docker-ce, builds went from 45 minutes to 3 minutes with no changes to our Dockerfile - fuse-overlayfs sits at 100% CPU all day, every day on our production servers - podman loses track of containers. Sometimes the…

I'm surprised it's using fuse-overlayfs instead of the overlay2 driver. Is it that you're using rootless containers in production? For rootful containers I think you'd have the same experience as docker here (or at least much closer).

Re: Podman: A Daemonless Container Engine

#178

Earlier quoted context omitted.

With root it's easy to mess up the system badly ("I'll just fix that small error") and/or let all systems slowly diverge. And if something fails on my workstation, it will fail on many others as well and needs a coordinated solution. Also, giving every dev root is a security liability, especially when there is absolutely no work reason that requires us to become root. So only a few "core IT" people have root access.…

Developing anything without root in your own machine these days must be an absolute nightmare. If I may ask, what stack are you using? I can imagine Java, but that's about it. Go, Python, Node, etc, all of them I've needed root access to test some packages, for instance. I would guess developers must be using a lot of "workarounds" without management's knowledge.

For Python, you can get most packages as non-root via `pip install --user $package` - but you are right, you might need root if these packages depend on system libraries (that they often wrap).

However, it maybe should not be a requirement. AFAIK nix allows you to install stuff as a normal user (and so does dnf on Fedora, but it depends on the package).

Re: Podman: A Daemonless Container Engine

#179
post #150
post #109

I've been trying to use podman in production and it is not working very well. I'm excited for this technology but it is not ready. - podman layer caching during builds do not work. When we switched back to docker-ce, builds went from 45 minutes to 3 minutes with no changes to our Dockerfile - fuse-overlayfs sits at 100% CPU all day, every day on our production servers - podman loses track of containers. Sometimes the…

What are the Github issue IDs for these 4 problems?

I agree with the sentiment that I should go find them. But I also agree with the sentiment that if RHEL 8 is going to make it hard to install docker and promote podman instead as a drop-in replacement, it's a bit frustrating.

Re: Podman: A Daemonless Container Engine

#180
post #164
post #161

Earlier quoted context omitted.

runc and containerd wouldn't exist if Redhat hadn't created the OCI specification and work around docker to make things interoperable. Docker, Inc was not playing ball and then redhat had to work around them. It wasn't a grudge it was just them keeping their business safe. The Open Containers Initiative was essentially designed to force Docker Inc to try to play better with others. They donated containerd to the OCI…

Your history is wrong. Docker created the OCI, not Red Hat. Docker donated both the spec and the runc implementation. Containerd was donated (also by Docker) to CNCF, a different organization, and I believe a few years later. You are correct that Docker did those things because of pressure to be more interoperable. Reading your linked tweet (“don’t mess with an engineering behemoth”) you seem to agree that Red Hat is…

Docker and Redhat were both founding members of OCI. CoreOS probably was responsible for a lot of the initial conversations. They wanted a standardized container runtime after finding some limitations in docker which caused them to create rkt. Docker inc wasn’t super into changing anything and as history as shown, wasn’t terribly into working well with others. The industry was going to move forward without them but they were offered a spot on the OCI if they played ball.

OCI would have made docker inc entirely irrelevant if they didn’t join it. They donated runc (I was wrong originally, dyslexia sucks sometimes) as the reference implementation so docker continued to stay at the forefront.

They donated a very minimal implementation of containerd to the CNCF when kubernetes wanted to do things with the container runtime interface (CRI) to make it more pluggable. As a more purpose built CRI, crio is better suited for k8s.

Docker did not create OCI any more than Redhat did. Redhat and a collection of other orgs did. Docker and Redhat were besties after Alexander Larson added device mapper support to docker. This is what allowed docker to run on any Linux distro that didn’t include the out of tree aufs. Look it up. Redhat then made a lot of money doing container multihost orchestration with openshift v3, which deprecated their own tool, geard. When Docker inc accepted a ton of VC money, they realized they actually had to monetize things. This is the part where the relationship started to seriously sour. The company that helped make them so popular was now suddenly a huge competitor. This is what lead to the problem.

It wasn’t a grudge from Redhat I mentioned in the linked tweet. It was docker inc thinking docker was anything other than commodity. It just is some very nice UX and cli tooling with a container registry around Linux namespace and Linux control groups. They just put it all together very nicely for developers to ship code faster. Podman is also a commodity. At its core, it just is some json parsing and pretty wrapping around Linux namespace and cgroups. Redhat knows this, and they don’t hide it. They monetize openshift and quay, not podman.

Sorry if this is a bit scrambled. Long posts on mobile are difficult.

Post reply on HN