Live data from Hacker News

Systemd vs. Docker

lwn.net

81–90 of 97 posts

Re: Systemd vs. Docker

#81

Earlier quoted context omitted.

That doesn't have anything to do with Poettering's quote. PR_SET_CHILD_SUBREAPER moves the ownership of an orphaned process to whichever process was selected rather than the default PID1, and that only works for descendant of the subreaper. The problem pointed by the quote is that normal software doesn't go around checking if it has zombie children and waiting on them, so in a container with random software S set as…

Yes it does. He's claiming that systemd should manage the container processes as pid1, because systemd will then clean up the zombies. But anything that reaps zombies can be pid1 -- systemd isn't special in this regard. And even if you did use something that didn't reap zombies as pid1, you could leverage PR_SET_CHILD_SUBREAPER as some other non-pid1 process to grab zombies for descendants it spawns. If you do use PR…

> Yes it does.

No, it still doesn't, sorry.

> He's claiming that systemd should manage the container processes as pid1, because systemd will then clean up the zombies.

The part that'a quoted only notes that PID1 is responsible for reaping orphaned zombies, that Random P. Application Process most likely doesn't do that, and that it causes problems.

> But anything that reaps zombies can be pid1 -- systemd isn't special in this regard.

The part you've quoted doesn't try to claim otherwise.

> And even if you did use something that didn't reap zombies as pid1, you could leverage PR_SET_CHILD_SUBREAPER as some other non-pid1 process to grab zombies for descendants it spawns.

That's a completely inane claim, the whole point of the article is the issue of people starting their application process as PID1, what are you suggesting, that applications should be modified to spawn an init which would use PR_SET_CHILD_SUBREAPER to which it would delegate spawning subprocesses? That's utter lunacy. Have some decency and regard for basic sanity and the context in which the quote appears.

> If you do use PR_SET_CHILD_SUBREAPER, then you need to reap whatever gets reparented to you; if you don't do this then the process table will eventually fill up with zombies. He is correct that few programs do that, but there's nothing that requires that to be done by pid1 if all the processes within the container are spawned by something that provides that functionality and uses PR_SET_CHILD_SUBREAPER.

Are you just making that hare-brained bullshit on the spot so that you don't have to admit your original comment was wrong?

What's the point of spawning a broken PID1 just so you can spawn a process using PR_SET_CHILD_SUBREAPER and doing the actual reaping correctly? Just spawn that as PID1 in the first place FFS.

Re: Systemd vs. Docker

#82
post #70
post #48

Earlier quoted context omitted.

> you really can't just debug a single machine in production when you have a cluster Somehow I ended up debugging, tracing, monitoring and even hotpatching individual machines in the cluster. Yeah the easy problems will show up in the monitoring and logs. The harder ones won't.

That must have been a pain in the butt :) . And for sure your right there are always exceptions. I guess I haven't ran into those issues probably because I run JVMs but I suppose if you have native code or an interpreter using native code I can see how it would be helpful to just SSH and figure out what the issue is. Now that I recall I have actually had to SSH a bunch of times because of Rackspace network interfaces…

It wasn't a huge pain. It was Erlang, so I could do those very easily. But it still had to be done by logging into a few machines and poking around. I can't imagine if that was somehow a bunch of C code combined with a kernel running in the same memory space.

Re: Systemd vs. Docker

#83

Earlier quoted context omitted.

Yes it does. He's claiming that systemd should manage the container processes as pid1, because systemd will then clean up the zombies. But anything that reaps zombies can be pid1 -- systemd isn't special in this regard. And even if you did use something that didn't reap zombies as pid1, you could leverage PR_SET_CHILD_SUBREAPER as some other non-pid1 process to grab zombies for descendants it spawns. If you do use PR…

> Yes it does. No, it still doesn't, sorry. > He's claiming that systemd should manage the container processes as pid1, because systemd will then clean up the zombies. The part that'a quoted only notes that PID1 is responsible for reaping orphaned zombies, that Random P. Application Process most likely doesn't do that, and that it causes problems. > But anything that reaps zombies can be pid1 -- systemd isn't special…

[deleted]

Re: Systemd vs. Docker

#84

Earlier quoted context omitted.

When you ask questions, especially leading ones, it causes a good deal of confusion around the topic at hand. The reasons behind this are complex, but they have something to do with our tendency to double bind each other. Someone has the right to say why something is "disqualified" for them, even if it is devoid of context. What is awesome here is that the leading expert for this topic is replying directly to the neg…

So in other words "questions are a burden and answers are a prison for oneself".

Regardless of their nature, questions are definitely a burden. However, I think the way some questions are put can cause a disproportionate amount of burden when they contain hidden meanings or agendas.

If someone is having issues being direct and use techniques to "hide" how they feel about something in a question, they effectively load the question with intent. I think sometimes those questions can be viral in nature, causing angry memes like what they mention in "This Video Will Make You Angry": https://www.youtube.com/watch?v=rE3j_RHkqJc

Logic would dictate that we should learn to avoid questions which cause excessive amounts of processing with little return in their answers. A simple way to filter on these is to ask if the question conflicts itself when answered in a given way.

Re: Systemd vs. Docker

#85

Earlier quoted context omitted.

Yes, but I will never have to use docker if I don't want to. For that matter, docker doesn't try to be cron, it doesn't want to handle mounting, didn't subsume udev, and doesn't encourage other project to link against it, and to drop all compatibility with non-linux systems. Systemd does, did, and is doing that right now.

"but I will never have to use docker if I don't want to." I predict that fairly soon install instructions for a lot of different types of software will be "docker run ..." For example, try running discourse without using docker.

And others will list systemd as a hard dependency. The future will be "fun".

Re: Systemd vs. Docker

#86
post #69

One of these is killing "zombie" processes that have been abandoned by their calling session. That's funny terminolgy, isn't it? Killing a process usually means sending it a signal, typically TERM or KILL, that causes it to exit. But a zombie process is one that has already exited, but hasn't been waited for by its parent, where its parent is either the process that spawned it, or if that process has died, the proces…

No, it's a zombie in the normal sense, the killing here is not sending it a signal but reaping zombie processes (in the sense of personified death reaping souls) by waiting on it. Things would probably be clearer if the quotes were around "killing" rather than "zombie", mayhaps the interviewer/writer was unfamiliar with the terminology.

I strongly doubt that! Josh knows the terminology. It was surly just an oversight.

Re: Systemd vs. Docker

#87
post #21
post #8

Earlier quoted context omitted.

Technology isn't that important here, it's what stems from it is. Adoption, infrastructure, tools, community. There's also a price for doing it differently. And believe you me, using BSD nowadays is the definition of doing things differently. What for? What I'm getting for losing my time and reinventing the tools that are already available and much more polished? Dockerfiles can be replicated. Docker Hub can be repli…

You do realise that "different" platforms can still be popular enough to have a lot of the same ecosystems. Hell, Docker itself used to be the "alternative"; not even that long ago in fact. FreeBSD might only have a fraction of the community that Linux does, but that's still a pretty large number of developers and sysadmins in real world terms. Disclosure: I run both FreeBSD and Linux systems.

There are also some interesting systems build on top of jails. I'm starting to use iocage, which uses both ZFS and jails to manage and create systems with great ease. Better or worse than Docker? It's different. It does some things docker does not, and vice versa. I'm currently in the evaluation stage, and it's likely I'll be using it to replace hand built jails running PostgreSQL, builds and other single-purpose tasks.

Re: Systemd vs. Docker

#88
post #16

Poettering says that PID 1 has special requirements. One of these is killing "zombie" processes that have been abandoned by their calling session. This is a real problem for Docker since the application runs as PID 1 and does not handle the zombie processes. For example, containers running the Oracle database can end up with thousands of zombie processes. Why does Poettering keep claiming this when he's the one who s…

I guess he's saying, that you can't just take any random binary and run it in a Docker container, because if that binary spawns a lot of children but does not wait for them, then you'll have a lot of zombies. Docker could run a minimal pid1 in each container to address this. Though if this had been a big issue I guess this would have been already fixed. Naturally, a proof of concept of the problem would be great. (Le…

See also the article linked earlier in the comments: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zomb...

Re: Systemd vs. Docker

#89
post #50

Earlier quoted context omitted.

Seeing how even the trivial pid1 "scripts" solve the problem, it's truly baffling why Docker doesn't have a --with-reaper flag. Also thanks for the Consul example, makes it much-much easier to see the issue and argue for a general solution. (So not every random app/project/service/daemon has to implement pid1 functionality.)

> Seeing how even the trivial pid1 "scripts" solve the problem, it's truly baffling why Docker doesn't have a --with-reaper flag. That doesn't fix the issue since you need to know about the issue and accept that it exists, at that point you can just as easily use one of the micro-inits available. The alternative is to enable it by default, but now you've broken BC for the weirdo who actually expects orphan processes…

I'm not worried about that. When operations people have problems they are rather quick to search and try solutions. But baking it into the Dockerfile is much more portable and automatic (from the operations point of view).

Re: Systemd vs. Docker

#90

Earlier quoted context omitted.

Integration is all well and good, I suppose. Until you put all your integration into a single process, which will kill your system if it ever crashes. Then I will have some words to say about you. Especially if you ever, ever, EVER, EVER, EVER, use assert, in ANY situation you could ever potentially recover from.

I disagree with your position on assertions and much prefer Bryan Cantrill's, because it results in bugs actually getting fixed: Hope is not a strategy, including for your software. If your state has become corrupt, it is incumbent upon you to die and donate your body to science, where it can be debugged.

Well, yes, in that case you should die, but take steps to make sure that your state doesn't become corrupt in the first place. Like, for instance, doing less, and making your service less complex.
Post reply on HN