In case anyone is interested, we open sourced a process manager and web interface a few months ago.
https://github.com/VividCortex/pm https://github.com/VividCortex/pm-web
Runsit – A process manager in Go
21–27 of 27 posts
Re: Runsit – A process manager in Go
#22Is there a site for this or some nature of docs? I'm still searching for a process manager that I can love for use with Docker (I've played with runit and am currently playing with s6), but the total lack of readme or docs makes this link fairly unhelpful.
Huh. I have somehow never heard of s6 but it looks very promising. What did you find when using it for this purpose?
I used s6 quite a bit in the past, it's got some pretty good improvements on daemontools. Back then I wrote some RPM specs to build s6 against musl: https://github.com/droyo/rpmbuild . I haven't tried to build them recently.
Re: Runsit – A process manager in Go
#23FWIW I also started writing a init-like server in Go. One thing I ran into was that Go's APIs sort of coerce you into having an extra thread per process. runsit also has this issue. See line 489 of runsit.go, pasted below. You can do it non-portably in Go by using os.ForkExec and Wait4(-1). The portable exec package assumes you will call Wait(pid), and not Wait(-1), which basically implies using a thread per process.…
We have an init process that runs inside docker containers, and we took a bit of a different route, in listening on SIGCHLD and then doing non-blocking Wait4(0,...) until there are no more children to reap:
Re: Runsit – A process manager in Go
#24FWIW I also started writing a init-like server in Go. One thing I ran into was that Go's APIs sort of coerce you into having an extra thread per process. runsit also has this issue. See line 489 of runsit.go, pasted below. You can do it non-portably in Go by using os.ForkExec and Wait4(-1). The portable exec package assumes you will call Wait(pid), and not Wait(-1), which basically implies using a thread per process.…
> I suppose Linux has an O(1) scheduler
Actually not anymore. The current CFS scheduler is no longer O(1) but O(logN):
Re: Runsit – A process manager in Go
#25FWIW I also started writing a init-like server in Go. One thing I ran into was that Go's APIs sort of coerce you into having an extra thread per process. runsit also has this issue. See line 489 of runsit.go, pasted below. You can do it non-portably in Go by using os.ForkExec and Wait4(-1). The portable exec package assumes you will call Wait(pid), and not Wait(-1), which basically implies using a thread per process.…
I wonder if it would make sense to make a "child poller" akin to the net poller: Have one goroutine loop forever: * wait for SIGCHLD * take the childlock * do nonblocking Wait() until no unwaited child remains * release the childlock The childlock would need to be taken for reading by os.Process.Kill and when a syscall that takes a PID of a child is called (after taking it we'd need to verify that the process we inte…
Yes, you can do that. As mentioned, it's not portable (which is fine with me).
However, you don't need to use goroutines (or threads). You do it as you would in C (and how all real PID 1 systems are written) -- with a single thread that starts processes, receives signals, and reaps children in a non-blocking fashion.
This style of program -- a program that needs to simultaneously wait for child processes/signals and fd events -- is quite awkward in Unix, but it definitely works when you get the idea.
To wait on a fd and a signal in a single threaded program, you would use the "self pipe trick" in classic Unix. In Linux, you can ask for a signal to be delivered over a file descriptor with fdsignal(). But AFAICT there is no real reason, and portability across Unix IS a good thing IMO (but not portability to completely different OS's like Windows; in that case I would write a completely separate program using their native APIs).
node.js actually does a great job making this API easy and efficient. It is probably the only runtime (Python/Ruby/JVM/etc.), that doesn't suffer from this problem doing "async processes" (i.e. a complement to async networking).
Re: Runsit – A process manager in Go
#26FWIW I also started writing a init-like server in Go. One thing I ran into was that Go's APIs sort of coerce you into having an extra thread per process. runsit also has this issue. See line 489 of runsit.go, pasted below. You can do it non-portably in Go by using os.ForkExec and Wait4(-1). The portable exec package assumes you will call Wait(pid), and not Wait(-1), which basically implies using a thread per process.…
We have an init process that runs inside docker containers, and we took a bit of a different route, in listening on SIGCHLD and then doing non-blocking Wait4(0,...) until there are no more children to reap: https://gist.github.com/burke/1c105378ac0629b39485
I think that is basically the same thing. I haven't used Wait4(0), but it looks like it is the same as Wait4(-1), as long as you don't change the process group ID of any of the children?
In any case, you are not calling Wait4(), which is what implies the thread per process.
Re: Runsit – A process manager in Go
#27Earlier quoted context omitted.
Huh. I have somehow never heard of s6 but it looks very promising. What did you find when using it for this purpose?
My initial thoughts were less than positive, just because building it is a less-than-stellar process. Part of that is because it has a couple of deps that must also be compiled (skalibs and execline), and part is because it follows the slashpackage conventions ( http://cr.yp.to/slashpackage.html ). Now that I've got it build, the next hurdle is making init scripts in execline. It's a fairly simple language, and I enj…
Note that:
- You do not have to follow the slashpackage convention to use or build s6. You can configure it out.
- You do not have to write your init scripts in execline. Any scripting language, including the shell, will do; I use execline because it allows me to do things /bin/sh does not, or not easily.
- There is a mailing-list, supervision@list.skarnet.org, dedicated to software like s6 and runit. If you have trouble building, installing or using such software, please subscribe and ask for help. I'm absolutely willing to make s6 easier to use, but I need the feedback.
Thanks for giving s6 a chance! You won't be disappointed - and if you are, make sure to let me know why.