Fork() without exec() is dangerous in large programs • Hacker News Board

Re: Fork() without exec() is dangerous in large programs

#101

Post by valarauca1 » Wed, Aug 17, 2016, 8:23 PM UTC

qwertyuiop924 wrote: post #99

valarauca1 wrote: post #94

Earlier quoted context omitted.

All I have to do is give up all hope of my code ever being portable to multiple architectures You know nothing about modern compiler atomics. Post 2011 compilers (LLVM, GCC, MSVC, ICC) standardized generic memory fences for C++ and C. These are fully portable as the compiler itself determines how the layout of Acquire/Releases needs to be changed on platform you are compiling too. Atomics are supported on ARM, x64, M…

Okay, what about HLLs? Memory fences don't work if you aren't writing C. And as I've said, forks are an easier way to get these guarantees, albeit at a moderate perf cost.

Seeing as this a conversation thread about raw system calls I assume we were working C not a HLL.

Are you just a professional contrarian?

Why are you concerned about system calls in a HLL?

Why are you concerned with concurrency guarantees in a HLL?

You care about none of these, you care about your run time. This is why you are working in a HLL.

Re: Fork() without exec() is dangerous in large programs

#102

Post by [deleted] » Wed, Aug 17, 2016, 9:02 PM UTC

valarauca1 wrote: post #101

qwertyuiop924 wrote: post #99

Earlier quoted context omitted.

Okay, what about HLLs? Memory fences don't work if you aren't writing C. And as I've said, forks are an easier way to get these guarantees, albeit at a moderate perf cost.

Seeing as this a conversation thread about raw system calls I assume we were working C not a HLL. Are you just a professional contrarian? Why are you concerned about system calls in a HLL? Why are you concerned with concurrency guarantees in a HLL? You care about none of these, you care about your run time. This is why you are working in a HLL.

[deleted]

Re: Fork() without exec() is dangerous in large programs

#103

Post by qwertyuiop924 » Wed, Aug 17, 2016, 9:08 PM UTC

valarauca1 wrote: post #101

qwertyuiop924 wrote: post #99

Earlier quoted context omitted.

Okay, what about HLLs? Memory fences don't work if you aren't writing C. And as I've said, forks are an easier way to get these guarantees, albeit at a moderate perf cost.

Seeing as this a conversation thread about raw system calls I assume we were working C not a HLL. Are you just a professional contrarian? Why are you concerned about system calls in a HLL? Why are you concerned with concurrency guarantees in a HLL? You care about none of these, you care about your run time. This is why you are working in a HLL.

  Are you just a professional contrarian?

No.

  Why are you concerned about system calls in a HLL?

Because in many HLLs, fork(2) is the best (or sometimes the only) option for concurrency. As in C, it is certainly the simplest.

Many things written in HLLs use fork. Unicorn is perhaps the most famous example.

  Why are you concerned with concurrency guarantees in a HLL?

Because if I have multiple threads of execution, I want them to run in parallel if possible, and I want to minimize the risk of deadlocks. fork(2) does both.

  You care about none of these, you care about your run time. This is why you are working in a HLL.

No. I care about them, as demonstrated above. And I don't know what you mean by care about my runtime. I would go so far as to claim that the above is an insult to me and everybody else who uses an HLL.

I care about getting stuff done. If C is the right tool for that, I'll use it. If an HLL is the right tool, I'll use it. And if syscalls are the right tool for the job, I'll use them, too.

Re: Fork() without exec() is dangerous in large programs

#104

Post by adrianratnapala » Thu, Aug 18, 2016, 1:22 AM UTC

eridius wrote: post #93

adrianratnapala wrote: post #16

Earlier quoted context omitted.

This is the very first time I have heard about fork-without-exec as dangerous, but it is not the first time I have heard about having to be careful about libraries and hidden threads. The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction.

> The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction. That's not true at all. Threads aren't that complicated, and there's lots of ways of using them that aren't complicated and don't leak anything. As a trivial example, if I want to process a bunch of data, it might be faster to process it in parallel, and I can use a thread t…

This is exaclty the kind of reasoning that makes library writers stumpble into using abstractions that turn out leaky. The whole point of the OP is to show one (of many) machanisms through which it does cause an observable change in behaviour.

And your example of a big calculation in a library is particurly bad, because the user can threadify that herself if she wants to.

    #include 
    #include 
    #include "my_eventloop.h"

    void launch_da_ting()
    {
          std::thread ting_doer([]{
              Result rez = yourlib::do_da_ting();
              my_event_loop::post([rez](){get_rez(rez);});
          });
          ting_doer.detach();
    }

Now it's true those five lines would become a single function call if your library forced the user to user a particular dispatching library. But I think that would be poor seperation of concerns.

Re: Fork() without exec() is dangerous in large programs

#105

Post by lmm » Thu, Aug 18, 2016, 9:09 AM UTC

MichaelGG wrote: post #91

lmm wrote: post #87

Earlier quoted context omitted.

> Global vars set wherever in a program are preferable to explicit params when creating a new process? The outside world is always going to be implicit global mutable state; spawning an arbitrary process to do who-knows-what is inherently that kind of problem. If you could express it as an actual function you wouldn't need to run a separate process. > And there's no way to have default params or anything? Well not in…

Well by that reasoning there's never any need for function purity since you can't eliminate global state on current computer designs. C easily allows you to have multiple functions. exec_with_current_state or something. And a struct is a way, too.

> Well by that reasoning there's never any need for function purity since you can't eliminate global state on current computer designs.

If you care about purity you don't use spawn-like functionality at all.

> C easily allows you to have multiple functions. exec_with_current_state or something.

That doesn't help. You'd need to have one variant where you just want to change the working directory, one variant where you just wanted to set an environment variable, one variant where you want to do both, one variant where you want to run as a different UID and change working directory but not touch the environment variables...

Re: Fork() without exec() is dangerous in large programs

#106

Post by eridius » Thu, Aug 18, 2016, 6:29 PM UTC

adrianratnapala wrote: post #104

eridius wrote: post #93

Earlier quoted context omitted.

> The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction. That's not true at all. Threads aren't that complicated, and there's lots of ways of using them that aren't complicated and don't leak anything. As a trivial example, if I want to process a bunch of data, it might be faster to process it in parallel, and I can use a thread t…

This is exaclty the kind of reasoning that makes library writers stumpble into using abstractions that turn out leaky. The whole point of the OP is to show one (of many) machanisms through which it does cause an observable change in behaviour. And your example of a big calculation in a library is particurly bad, because the user can threadify that herself if she wants to. #include #include #include "my_eventloop.h" v…

It's leaky in one extremely minor way that affects almost no programs whatsoever. It's extremely rare for anyone to do fork-without-exec, and anyone doing that already has a large number of restrictions they must be aware of.

And no, the user can't threadify that themselves. Your sample code is putting the library call on a thread. That's not at all what I was talking about. I was talking about a library using multiple threads (or, more likely, some abstraction over a thread pool that handles scheduling chunks of work on threads for you, such as libdispatch on Apple platforms) in order to process a bunch of data in parallel, data that only the library knows about. The user makes a single call into the library, the library processes a bunch of data in parallel, and then returns to the user. From the user's perspective it's a synchronous call, but it runs in a fraction of the time it would if the library wasn't allowed to use threading internally.