Live data from Hacker News

A fork() in the road

microsoft.com

171–180 of 184 posts

Re: A fork() in the road

#171

While fork() might be sub-optimal for launching different programs (fork() + exec() vs. posix_spawn()), it's absolutely essential in several types of common systems that don't use it to launch different programs. Fork-requiring program class 1: The biggest example where fork() is needed are webservers/long-running programs with significant unchanging memory overhead and/or startup time. Many large applications writte…

> While fork() might be sub-optimal for launching different programs (fork() + exec() vs. posix_spawn())

I don't think it is suboptimal. As the paper acknowledges it primary use is to set up the environment of the program you are about to exec(). There are four points to be made about that:

1. If you don't need to set up the environment it imposes almost no coding overhead. It reduces to "if (!(pid = fork()) exec(...)". That's hardly a huge imposition.

2. It doesn't seem to impose much runtime overhead either. If it did Linux and BSD would have acquired a spawn() syscall's ages ago. As it is they all implement posix_spawn() using a vfork() / exec(). Given we are talking a 30 year history here any claims getting rid of the fork() would give a noticeable performance boost should not be taken seriously without evidence.

3. If you do need to setup the environment then yes there are traps with threads and other things. As the paper says it's terrible - but to paraphrase Churchill the one thing it has in it's favour is it's better than all the other ways of doing the same thing. They actually acknowledge how to replace flexibility allowed by fork() is an open research question. "We think it's horrible, but we don't have an alternative" isn't a convincing argument.

4. For all it's faults fork() has one outstanding attribute - it's conceptually drop dead simple: "create an exact copy of the process, the sole difference being getpid() returns a different value". That translates to bugger all code needed to implement it, few bugs, small man pages and a simple interface. A replacement providing the same flexibility will be some hideously complex thing that tries to implement all the use cases people used fork() for. It will be big and hard to learn, hard to use correctly, take reams of code, still won't do all that fork() allowed you to do. We will be complaining about if for decades to come.

I stopped reading the paper when they claims O_CLOEXEC was an overhead imposed by fork(). It isn't. The telltale give away should be it doesn't take effect on a fork() - it happens on the exec(), and the spawn() or whatever does exec()'s job. If you remove fork() things like O_CLOEXEC is your only way to control what environment your child process gets. Therefore one outcome of removing fork() is the reverse of what they claim - you won't get less O_CLOEXEC's, you will get many, many more of them as programmers clamour for ways to do the things fork() allowed them to do.

Re: A fork() in the road

#172
post #110

Earlier quoted context omitted.

as a function obviously, the point is that it does not compose easily with other abstractions. That is every other library and OS functionality needs to be fork-aware. spawn do not have this requirement.

The concept of "fork aware" didn't exist until threads. You could argue it's a thread problem. Remember, every library and OS functionality aso needs to be "thread aware" when threads are introduced. The pthread_atfork function can be thought about as "what do we do about thread and thread paraphernalia when we fork" rather than "what do we do about fork when we have threads". Even the close-on-exec flag race conditi…

> You could argue it's a thread problem

But I explicitly want to not do it :) thread are obviously a good thing to have.

> every library and OS functionality aso needs to be "thread aware"

which is good, because differently from the case with fork thread aware libraries/OS help performance. Fork aware libraries/OS (in the case fork+exec) do not.

Re: A fork() in the road

#173

Earlier quoted context omitted.

Specifically, you'd want to do something like: pid_t child = pfork(); for(int fd=0;fd (Y'know, this looks kind of familiar...)

I'm really not sure what you're implying, can you please state it explicitly? I'd expect this code to look similar to both the CreateProcess and fork models, at the very least.

a1369209993's example snippet appears to be what is implied from section 6 ("REPLACING FORK"), subsection "Low-level: Cross-process operations":

> clean-slate designs [e.g.,40, 43] have demonstrated an alternative model where system calls that modify per-process state are not constrained to merely the current process, but rather can manipulate any process to which the caller has access. This yields the flexibility and orthogonality of the fork/exec model, without most of its drawbacks: a new process starts as an empty address space, and an advanced user may manipulate it in a piecemeal fashion, populating its address-space and kernel context prior to execution, without needing to clone the parent nor run code in the context of the child.

Re: A fork() in the road

#174

Earlier quoted context omitted.

I'm really not sure what you're implying, can you please state it explicitly? I'd expect this code to look similar to both the CreateProcess and fork models, at the very least.

a1369209993's example snippet appears to be what is implied from section 6 ("REPLACING FORK"), subsection "Low-level: Cross-process operations": > clean-slate designs [e.g.,40, 43] have demonstrated an alternative model where system calls that modify per-process state are not constrained to merely the current process, but rather can manipulate any process to which the caller has access. This yields the flexibility an…

> a1369209993's example snippet appears to be what is implied from section 6 ("REPLACING FORK"), subsection "Low-level: Cross-process operations"

Oh definitely. But why are they saying it "looks kind of familiar..."? That subsection is already the subject of the conversation. Surely they're not saying it looks similar to itself, right?

Re: A fork() in the road

#175

Earlier quoted context omitted.

a1369209993's example snippet appears to be what is implied from section 6 ("REPLACING FORK"), subsection "Low-level: Cross-process operations": > clean-slate designs [e.g.,40, 43] have demonstrated an alternative model where system calls that modify per-process state are not constrained to merely the current process, but rather can manipulate any process to which the caller has access. This yields the flexibility an…

> a1369209993's example snippet appears to be what is implied from section 6 ("REPLACING FORK"), subsection "Low-level: Cross-process operations" Oh definitely. But why are they saying it "looks kind of familiar..."? That subsection is already the subject of the conversation. Surely they're not saying it looks similar to itself, right?

Ahh. I thought it was familiar because it was the way one would write that sort of code now, in a fork/exec environment, only replacing the call to change local information into ones to change the child's information.

Re: A fork() in the road

#176
post #157

Earlier quoted context omitted.

The Linux kernel doesn't crash much, unless you have dodgy drivers or dodgy hardware. Whether your userland programs crash or not depends on what you're running. I don't expect to see sshd crashing, for example, though it's true that almost any program will exit suddenly if the system runs out of memory, which to an ordinary user looks like a crash, though it's a very different thing really.

If you weren't talking about userland crashes, then your crack about "an OS on which crashing is expected behavior" makes no sense.

The comment was not meant to be taken all that seriously, of course, but an OS is more than just the kernel, and I do tend to disapprove of brushing a crash under the carpet.

System runs out of memory, various processes get terminated, and the easiest way to get it back into a good state is a restart: not that worrying, but do you have a memory leak? Some process segfaults with 54584554454d4f53 in the PC: should be investigated, not glossed over.

Re: A fork() in the road

#177

Earlier quoted context omitted.

This may be true, but I don't want MS having ANY say on what goes into a Linux or FreeBSD OS. None. They have an agenda that doesn't fit in well with FOSS. May no mistake about it, driving everything so that it works with Azure/VS, whatever, is about staying relevant in a world that is largely leaving them behind. Short of having to write PS at work (required), I haven't run anything MS at home since 1998 and have no…

> This may be true, but I don't want MS having ANY say on what goes into a Linux or FreeBSD OS. None. They have an agenda that doesn't fit in well with FOSS. False. MS is now one of the leading FOSS contributors. You're living in the past.

My living in the past is YOUR opinion. There are many millions of FOSS users who are highly opposed to MS having any influence whatsoever on FOSS. They have an agenda and it's never in the best interest of the FOSS crowd. Do you think they are doing what they do out of benevolence? It's done for MS software compatibility with FOSS, so users will choose Azure and their other cloud offerings. It's done purely to keep them in the game and relevant. No other reason. I heavily distrust MS, as over the years they have given many reasons not to trust them. Ever wonder why so many people abandoned GitHub after MS bought them?

Re: A fork() in the road

#178

Earlier quoted context omitted.

> Part of the problem is the DLLs, as many have mentioned, and also the fact that each statically links in its own CRT (C run-time) No, that isn't the case on DLLs shipped with Windows.

But it is for 3rd party DLLs.

Well, that depends on the DLL.

Re: A fork() in the road

#179

Earlier quoted context omitted.

That has security considerations when spawning a process to run an executable that gets privilege on exec (think set-uid on Unix).

Easy to deal with by not applying privileges until the parent is done tinkering and hits start.

I don't see how. The privilege elevation mechanism cannot apply to an already-running process, since then there will be a way to subvert the process.

Now, the better answer to that is to have nothing like a set-uid mechanism, which would be nice, for sure. But just how much violence are we to do to the Unix model, and when are we expected to finish this? It's not like Linux can be abandoned -- for better or worse, Linux "won".

Re: A fork() in the road

#180

Earlier quoted context omitted.

Easy to deal with by not applying privileges until the parent is done tinkering and hits start.

Specifically, you'd want to do something like: pid_t child = pfork(); for(int fd=0;fd (Y'know, this looks kind of familiar...)

vfork() is the right tool. I dunno what semantics you have in mind for p_execve().
Post reply on HN