Fork() without exec() is dangerous in large programs • Hacker News Board

Re: Fork() without exec() is dangerous in large programs

#51

Post by lmm » Wed, Aug 17, 2016, 11:18 AM UTC

gpderetta wrote: post #35

chubot wrote: post #21

Earlier quoted context omitted.

You can have your cake and eat it too -- there's no throwing away of performance gains. Libraries can use threads, but not directly or "silently". They should take threads as parameters. Then the application knows when they're being used and the application programmer can reason about it.

compilers and runtimes can spawn threads implicitly when required (think of openmp, cilk+, or even autopar). Are you saying that libraries shouldn't use these facilities?

If you're spawning threads implicitly then you're no longer a library, you're a framework. Sometimes a framework is a good fit, but frameworks don't compose (if you try to use two frameworks in the same application you're gonna have a bad time) so there is a cost. But in any case, better a framework that's explicitly a framework than a "library" which can't actually be used freely in general-purpose code.

Re: Fork() without exec() is dangerous in large programs

#52

Post by lmm » Wed, Aug 17, 2016, 11:22 AM UTC

the_mitsuhiko wrote: post #27

ahoka wrote: post #18

Earlier quoted context omitted.

I would like to see the reasons this comment was down-voted. It is true that fork(2) is simple and elegant design, but it involves a huge accidental complexity and hidden implications like in the original article. There is a reason that Linux has clone(2) underneath.

> It is true that fork(2) is simple and elegant design What exactly is simple and elegant about it? Have you looked into how much tooling is necessary everywhere in unix to make it work? It's insane. It has a huge footprint and it does not provide standardized APIs to make it work for non covered cases (the best we have is pthread_atfork which is not portable).

> What exactly is simple and elegant about it?

Spawn-style APIs have to take a billion parameters that are mostly set to defaults, for things like working directory, environment variables, user ID, controlling terminal. Whereas fork+exec means you can express these things in a more compositional, buildery style: fork, change the two things you actually need to change, then exec.

Re: Fork() without exec() is dangerous in large programs

#53

Post by ahoka » Wed, Aug 17, 2016, 11:30 AM UTC

the_mitsuhiko wrote: post #27

ahoka wrote: post #18

Earlier quoted context omitted.

I would like to see the reasons this comment was down-voted. It is true that fork(2) is simple and elegant design, but it involves a huge accidental complexity and hidden implications like in the original article. There is a reason that Linux has clone(2) underneath.

> It is true that fork(2) is simple and elegant design What exactly is simple and elegant about it? Have you looked into how much tooling is necessary everywhere in unix to make it work? It's insane. It has a huge footprint and it does not provide standardized APIs to make it work for non covered cases (the best we have is pthread_atfork which is not portable).

Elegant in a 'clever trick' sense, not as good engineering.

Re: Fork() without exec() is dangerous in large programs

#54

Post by evanj » Wed, Aug 17, 2016, 11:38 AM UTC

zzzcpan wrote: post #50

> How to use fork safely: 1. Only use fork to immediately call exec. 2. Fork a worker at the beginning of your program, before there can be other threads. 3. Only use fork in toy programs. 4. Stop writing broken multithreaded code altogether or if you must at least run an event loop per core/thread and use a wrapper for fork() to put the system into a fork()able state before forking. It's nice and reliable. Multithre…

You are completely correct: it is in fact possible to use fork if you can carefully control the state of threads in your program. My point is that can be difficult in large software projects, particularly when random APIs like "get the system proxy settings" use threads without you having any ability to control them.

Re: Fork() without exec() is dangerous in large programs

#55

Post by qznc » Wed, Aug 17, 2016, 11:42 AM UTC

the_mitsuhiko wrote: post #46

qznc wrote: post #45

Earlier quoted context omitted.

Fork is simple, but not easy.

What makes for simple? I really need to understand this. I implemented fork many years ago for a university operating system and nothing about either the implementation nor the usage is simple. I use fork on a daily basis and the POSIX version of it is insanely complex for everybody involved.

> I implemented fork many years ago for a university operating system and nothing about either the implementation nor the usage is simple

Me too. I found it simple: Create new child process, copy the address space, copy the instruction pointer, let it run.

If you want to improve performance via copy-on-write, it gets more complex to implement. If you combine it with other features like threads, mmap, limits, etc use becomes complex, but the mechanism itself is simple.

You might be able to design an easy variant of fork, which handles threads, mmap, limits, etc in a sane way, but that would not be simple anymore.

Fork is actually a good example, why simple and easy are different concepts and why it is really hard to achieve both.

Re: Fork() without exec() is dangerous in large programs

#56

Post by evanj » Wed, Aug 17, 2016, 11:46 AM UTC

hyperbovine wrote: post #49

I ran into this exact problem recently in a multithreaded Python app and spent two days trying to figure out wtf was going on. The multiprocessing spawn startup mode that was added in 3.4 solves this for most use cases at the expense of a small performance hit. For 2.x you are SOL however.

Part of my motivation for writing these things is because after I've wasted so much time, I'd love to help others not do the same. Hopefully the article shows up when you search for the right error message. Its also so I remember what the heck was happening.

Re: multiprocessing in 3.4: I think it is unfortunate that due to backwards compatibility, the "forkserver" mode can't become the default on Unix, since it would help avoid this particular issue.

Re: Fork() without exec() is dangerous in large programs

#57

Post by evanj » Wed, Aug 17, 2016, 11:53 AM UTC

antirez wrote: post #36

If you are in control of all the threads that an application is running, you can call fork() safely by making sure the threads are put in a safe state (no critical locks retained) when the fork() call happens. Also note that many things that should normally be unsafe, like having running threads calling malloc() while another thread is forking, are actually safe in the real world using certain implementation of fork,…

You are completely correct! The thing that surprised me is how difficult that can be. For example, on Mac OS X if you want to read the system proxy settings, you magically get threads added to your program that you can't control.

Re: Fork() without exec() is dangerous in large programs

#58

Post by rlpb » Wed, Aug 17, 2016, 11:54 AM UTC

chubot wrote: post #5

caf wrote: post #2

In my opinion, it's libraries that secretly use threads under the covers (but still access state shared with the rest of the program like the malloc heap) that are what's dangerous.

Yes. Libraries should provide algorithms and data structures, and parameterize a threading interface if necessary (though I think it's usually not necessary). Applications can then instantiate their own threads and pass them to the library (i.e. dependency injection.) So if you have 10 different concurrent libraries in an application, then you don't end up with 10 different threading paradigms or 10 different thread…

> Threads are inherently the right way to do concurrency for more complex tasks.

True, but they're more often used as a crutch where a select-style loop and/or state machine would be more suitable, leaving race conditions in their wake.

Re: Fork() without exec() is dangerous in large programs

#59

Post by valarauca1 » Wed, Aug 17, 2016, 11:54 AM UTC

zzzcpan wrote: post #50

> How to use fork safely: 1. Only use fork to immediately call exec. 2. Fork a worker at the beginning of your program, before there can be other threads. 3. Only use fork in toy programs. 4. Stop writing broken multithreaded code altogether or if you must at least run an event loop per core/thread and use a wrapper for fork() to put the system into a fork()able state before forking. It's nice and reliable. Multithre…

Fork/Exec is only good if you are going to run a different program then you are currently executing.

If you want a series of worker threads that are effectively the same as the master/initial thread. The proper way to do this clone(2) not fork, or fork/exec.

When used properly clone allows your group of threads to share a single PID, and TGID (Thread Group IDentifier). This cuts down on kernel resources your process(es) are using. They can natively share file descriptors and memory between each other (improving cache coherence). Also their signals are handled globally for all threads at once, not each thread managing it's own signals like Fork/Exec will result in.

Re: Fork() without exec() is dangerous in large programs

#60

Post by evanj » Wed, Aug 17, 2016, 11:56 AM UTC

kostyash wrote: post #43

It sounds like if you use threads and fork it might be a disaster. Its true, but the author blames only fork and skip the second part of the problem, threads. This is not fair, because fork is much older than POSIX threads. In fact, POSIX threads were poorly designed to use with fork.

Fair point. My opinion is that in today's age of multi-core CPUs, shared memory concurrency is extremely useful for making efficient use of computing resources. As a result, I find threads to be unavoidable in most large systems I've dealt with recently.