Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

311–320 of 358 posts

Re: Moving beyond fork() + exec()

#311

Post by netbsdusers »

dcrazy wrote: post #125
nvme0n1p1 wrote: post #72

Earlier quoted context omitted.

That's not the reason for the performance difference. Windows does have a fork primitive (ZwCreateProcess) and it's still slower than Linux's equivalent.

Again, NtCreateProcess does not implement fork(). The fundamental characteristic of fork is that the child is an exact replica of the parent, down to the instruction pointer. Windows does not have a way to create a process object with such a configuration. Also, using the Zw prefix doesn’t make you look more knowledgeable, it makes you look like you’re trying way too hard to borrow credibility.

It's a completely uncontroversial fact that NT does implement fork(). Turn to page 183 of Helen Custer's "Inside Windows NT" and you will read about it.

Re: Moving beyond fork() + exec()

#313

Post by dwattttt »

uecker wrote: post #294
amluto wrote: post #250

Earlier quoted context omitted.

Help me out here, please. Off the top of my head, the exec command is dependent on exec, except that a spawn + wait implementation would be a mostly okay substitute. Pipes and redirections don’t need fork + exec. Neither do subshells.

If you use pipe() you get two ends in the same process, then you fork and child and parent can communicate. This is how a unix shell setups up pipes and it is rather elegant.

Doing the same thing on Windows, I create the pipe and get two ends in the same process. Then I'd call CreateProcess and indicate I want the pipe's handle (fd) inherited to the child, and I'd use a prearranged way to tell the child what the fd value is it should use.

Possibly the most common way to tell the child the value is by setting it as a CLI arg in CreateProcess.

Re: Moving beyond fork() + exec()

#314

Post by fonheponho »

uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

> The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs.

Unfortunately, the opposite is true, when the parent process is multi-threaded. In the child process, only one thread exists (the thread returning from fork()), but the memory is an exact copy of the parent's. As a result, the child may inherit locks (resident in memory) that are in acquired state, but have no owner threads -- the threads that are responsible for eventually releasing those locks in the child's copy of the process memory do not exist in the child. If the single thread in the child process (returning from fork()) attempts to take such a lock (before exec), it deadlocks. This is why POSIX says that only async-signal-safe functions may be called in a child process, between fork and exec. And then, for example, "malloc" is not such a function (at least per POSIX), so the fork-to-exec environment in the child process is an extremely uncomfortable one. You've got to preallocate everything in the parent, can't report errors to stderr, etc.

https://pubs.opengroup.org/onlinepubs/9799919799/functions/f...

https://pubs.opengroup.org/onlinepubs/9799919799/functions/V...

The fork(2) Linux manual page spells out the sam restriction.

https://man7.org/linux/man-pages/man2/fork.2.html

https://man7.org/linux/man-pages/man7/signal-safety.7.html

"pthread_atfork" exists, but is effectively unusable.

https://pubs.opengroup.org/onlinepubs/9799919799/functions/p...

Re: Moving beyond fork() + exec()

#315

Post by jerf »

dcrazy wrote: post #179
jerf wrote: post #176

Earlier quoted context omitted.

Who said anything about it being a "bad idea"? I also explicitly said this wasn't unsolvable. My point isn't about technical implementations or code, my point is that the casual "I want to share nothing about the parent process" thought in sanderj's mind, and presumably a lot others, is much more ill-defined than they realize. There's a lot more state that a process has than what file descriptors are open in a modern…

Linux posix_spawn is a wrapper around clone and exec. There is no primitive on Linux to create an entirely blank process. This is adequately discussed in the linked LWN post. Other operating systems either have parallel APIs to fork (e.g. the posix_spawn syscall on macOS) or do not provide fork at all (Windows).

You seem to persist in reading into my words claims that aren't there and then excitedly debunking them. I feel I'm extraneous to this process, though, so I think I'll let you carry on arguing with the guy in your head on your own terms. It's more fun for both of us.

Re: Moving beyond fork() + exec()

#316

Post by uecker »

fonheponho wrote: post #314
uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

> The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Unfortunately, the opposite is true, when the parent process is multi-threaded. In the child process, only one thread exists (the thread returning from fork()), but the memory is an exact copy of the parent's. As a result, the child may inherit locks (resident in memory) that are in acq…

Yes, threads are a complication, but this still not "the opposite".

Re: Moving beyond fork() + exec()

#317

Post by uecker »

dwattttt wrote: post #313
uecker wrote: post #294

Earlier quoted context omitted.

If you use pipe() you get two ends in the same process, then you fork and child and parent can communicate. This is how a unix shell setups up pipes and it is rather elegant.

Doing the same thing on Windows, I create the pipe and get two ends in the same process. Then I'd call CreateProcess and indicate I want the pipe's handle (fd) inherited to the child, and I'd use a prearranged way to tell the child what the fd value is it should use. Possibly the most common way to tell the child the value is by setting it as a CLI arg in CreateProcess.

Yes, and CreateProcess needs special facilities to make this possible while in the UNIX model you don't.

Re: Moving beyond fork() + exec()

#318

Post by m132 »

sanderjd wrote: post #264
m132 wrote: post #254

Earlier quoted context omitted.

Node, Python, PowerShell, and the rest do (almost) just that. launchd and systemd famously strived to remove as much shell from the start up process as possible because it was harming boot times and introducing unpredictability.

I don't know Node or PowerShell very well, but I'm not sure what you mean by this with respect to Python.

CPython doesn't usually create subprocesses unless specifically asked to, it loads Python modules and native extensions into its process. The former is similar (you're still extending an existing process with new code, just interpreted), the latter is literally dlopen(), so loading dynamic libraries.

A lot of other Python implementations don't have the ability to spin up new processes at all too.

Re: Moving beyond fork() + exec()

#319

Post by uecker »

IshKebab wrote: post #298
uecker wrote: post #233

Earlier quoted context omitted.

A lot of features of UNIX shells are build around pipe and dup and the fork + exec model. One can certainly implement in differently, but it is - like UNIX in general - very nice and elegant.

It's an elegant hack, but it's still a hack. Not what we should be doing in 2026.

I would prefer to continue to use elegant interfaces even beyond 2026.

Re: Moving beyond fork() + exec()

#320

Post by tliltocatl »

mort96 wrote: post #239
pjmlp wrote: post #235

Earlier quoted context omitted.

It actually kind of is, hence why you have information about parent/child and get to share memory. This is how a http server back in the day would share the request context for the child process to reply back.

I would say that pipes and shared memory are the IPC mechanisms? Controlling the state of the exec'd process's file descriptors would counts as a way to set up interprocess communication, but once that's done, it's the pipe or SHM that does the actual communication.

The problem with POSIX IPC is that passing file descriptors between processes (other than parent passing to child via fork) is hard. Yes, SCM_RIGHTS can do it, but it is quite error prone and rarely done.
Post reply on HN