Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing dumb-init, an init system for Docker containers

engineeringblog.yelp.com

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Introducing dumb-init, an init system for Docker containers

#21

Post by justinsaccount »

zenlikethat wrote: post #20
justinsaccount wrote: post #17

Earlier quoted context omitted.

I kinda solve that by doing something like this: # install runtime deps dpkg -l | awk '{print $2}' | sort > old.txt # install build deps # build software dpkg -l | awk '{print $2}' | sort > new.txt apt-get -y remove --purge $(comm -13 old.txt new.txt) Probably a better way to accomplish that, but it was the easiest way I could see to implement an 'undo'

If doing this sort of thing, make sure to accomplish it in a single step (image layer) in the Dockerfile. Otherwise you won't be doing any good as the "removed" files actually persist behind a layer which specifies them as removed.

I've been using docker-squash for that. This way I can take advantage of layer caching during development, and still have a small image for uploading.

Re: Introducing dumb-init, an init system for Docker containers

#22

Post by akavel »

If I understand correctly, the main goal here can be summarized in the quote below:

"The motivation: modeling Docker containers as regular processes

[...] we want processes to behave just as if they weren’t running inside a container. That means handling user input, responding the same way to signals, and dying when we expect them to. In particular, when we signal the docker run command, we want that same signal to be received by the process inside."

and that seems to me as the core reason why they can't just use a simple init system (like e.g. runit I suppose?)

Re: Introducing dumb-init, an init system for Docker containers

#23

Post by vacri »

rvense wrote: post #12
jarito wrote: post #7

Earlier quoted context omitted.

Generally accepted practice is no - they aren't worth having. Containers should only contain a single process. That process shouldn't be writing logs to disk (hence no logrotate) and timed tasks would generally be done outside the container rather than in it (though there are a lot of ways to skin that cat). Single process containers generally don't need all the baggage of a full init system or other dependencies - h…

At my current job, we're basically using Docker as a sort of package manager and deployment script runner. Our containers are very fat, things are installed with apt. One of them has GCC in it but I'm not sure why. One installs Node and runs a few js scripts during the build process, then never runs it again but keeps it around. It's obviously wrong, but I think it's just a new set of bad ideas that this software has…

> One of them has GCC in it but I'm not sure why.

'npm install' needs 'make' half the time. Sometimes it's easier on debian-based setups to install build-essential than make, as it will pull in a few other things that help as well. GCC is one of those things - might it be that that container has build-essential installed?

I used to run docker with fat containers, and have now just finished getting rid of docker in favour of .debs. We were basically using it as a package manager, and it is terrible at the job. Docker has its use-cases, but package management isn't one of them. The docker tagging system is particularly bad at the job.

Re: Introducing dumb-init, an init system for Docker containers

#24

Post by Gigablah »

stephen-mw wrote: post #2

Very nice! I see this as the next evolution to the phusion's custom init system[0], which was created to solve largely the same problems. I should be able to take Yelp's dumb-init and add it easily to any linux container I want -- including things such as Alpine[1] [0] https://github.com/phusion/baseimage-docker/blob/master/imag... [1] https://github.com/gliderlabs/docker-alpine

s6 and s6-overlay were also created with the same goal: https://github.com/just-containers/s6-overlay

I use it in my Alpine containers.

Re: Introducing dumb-init, an init system for Docker containers

#25

Post by jzelinskie »

gnud wrote: post #3

The actual reason you really need a proper PID 1 is not explained in this post, but a couple of clicks away at [0]: >[...] the init process must also wait for child processes to terminate, >before terminating itself. >If the init process terminates prematurely then all children are terminated uncleanly by the kernel. 0: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zomb...

Considering phusion/baseimage has been around for more than 2 years and plenty of people have been using an init system inside their containers that contain multiple process, why didn't Yelp just pick something up off the shelf? Why not use runit or one of the plenty of more mature lightweight init systems?

Re: Introducing dumb-init, an init system for Docker containers

#26

Post by yrro »

XorNot wrote: post #14
rvense wrote: post #12

Earlier quoted context omitted.

At my current job, we're basically using Docker as a sort of package manager and deployment script runner. Our containers are very fat, things are installed with apt. One of them has GCC in it but I'm not sure why. One installs Node and runs a few js scripts during the build process, then never runs it again but keeps it around. It's obviously wrong, but I think it's just a new set of bad ideas that this software has…

Under a time crunch I've not found a way to use language-package managers and not wind up with gcc in the container. The problem is apt does a poor job of letting you setup something like build-essential and then remove it and leave just the runtime shared libraries you need for the other things you build to actually work.

If you installed build-essential, then removed it, then apt-get --purge autoremove should remove the packages that build-essential pulled in that were not already installed.

Re: Introducing dumb-init, an init system for Docker containers

#27

Post by DanielDent »

Another alternative, for what it's worth: https://github.com/rciorba/pidunu

I've created an Ubuntu PPA packaging of it (https://launchpad.net/~danieldent/+archive/ubuntu/pidunu) and you can see an example of it in use at: https://github.com/DanielDent/docker-powerdns

For situations involving multiple processes, there's also https://github.com/just-containers/s6-overlay

Example use: https://github.com/DanielDent/docker-nginx-ssl-proxy (automated Let's Encrypt SSL front-end)

Re: Introducing dumb-init, an init system for Docker containers

#28

Post by vikiomega9 »

ckuehl wrote: post #18
vikiomega9 wrote: post #16

> Having a shell as PID 1 actually makes signaling your process almost impossible. Signals sent to the shell won’t be forwarded to the subprocess, and the shell won’t exit until your process does. The only way to kill your container is by sending it SIGKILL (or if your process happens to die). Noob question. Why is it impossible? You have the PID, no?

Good question! The problem is trying to signal it from outside the Docker container. If your container has a process tree like PID 1: /bin/sh +--- PID 2: then if you use `docker signal` from the host, it will only send a signal to PID 1, which is the shell. However the shell won't forward it on to your Python server, so nothing happens (in most cases). dumb-init basically replaces the shell in that diagram, but forwa…

Ah that makes sense. I did not realize how docker-signal forwarded signals. From its perspective using PID 1 makes sense because that's where the "application" should run as specified in your dockerfile.

Re: Introducing dumb-init, an init system for Docker containers

#30

Post by predakanga »

jzelinskie wrote: post #25
gnud wrote: post #3

The actual reason you really need a proper PID 1 is not explained in this post, but a couple of clicks away at [0]: >[...] the init process must also wait for child processes to terminate, >before terminating itself. >If the init process terminates prematurely then all children are terminated uncleanly by the kernel. 0: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zomb...

Considering phusion/baseimage has been around for more than 2 years and plenty of people have been using an init system inside their containers that contain multiple process, why didn't Yelp just pick something up off the shelf? Why not use runit or one of the plenty of more mature lightweight init systems?

I can't speak to the others that have been mentioned in this thread (tini in particular seems to be identical), but the solution used by phusion/baseimage is written in python[0] - a C-based solution allows for lighter-weight containers

[0]: https://github.com/phusion/baseimage-docker/blob/master/imag...

Post reply on HN