Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

161–170 of 358 posts

Re: Moving beyond fork() + exec()

#161

Post by dcrazy »

zozbot234 wrote: post #78
pjmlp wrote: post #70

Earlier quoted context omitted.

Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.

Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.

NT was designed to be platform-agnostic, and its original target was the DEC Alpha. Its process model owes nothing to pre-386 CPUs. The WinAPI CreateProcess function is a layer atop NtCreateProcess, so that is where the pre-386 heritage lives. But even the WinAPI process model changed significantly with 32-bit Windows.

Re: Moving beyond fork() + exec()

#162

Post by uecker »

jcranmer wrote: post #144
uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

Calling that elegant is a path dependence of the history of fork+exec. In an alternative world where fork+exec never existed, a lot of those "usual APIs" would probably have had an explicit pid argument to them that let you modify process configuration from a different process. (This is how Fuschia works, e.g.). There's a lot of benefit to this world: the most obvious is that you don't have to magic up some IPC syste…

Weren't there enough parallel paths of development in this world?

Re: Moving beyond fork() + exec()

#163

Post by mort96 »

foresto wrote: post #153
mort96 wrote: post #79

Earlier quoted context omitted.

The problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1, and you need overcommit. Now these decisions aren't objectively bad , but they have significant trade-offs and it's probably not a go…

> The problem with fork isn't really that it's slow. Did someone suggest that it was?

anarazel's comment focuses entirely on performance, indicating that they have an impression that the discussion about why fork is bad is about performance. I'm not entirely sure where this impression came from, as it's not mentioned in rom1v's quote nor a point in the linked paper, "A fork() in the road".

Re: Moving beyond fork() + exec()

#164

Post by mort96 »

pjmlp wrote: post #149
sunshowers wrote: post #140

Earlier quoted context omitted.

The problem is that threads are not fault boundaries but processes are. So they're not interchangeable when you care about resilience and misbehaving code.

True, but on Windows the approach is then to use COM servers, which have a faster IPC model, and can even serve multiple clients, depending on how the appartement space is configured.

"Faster IPC model" than what? Faster than writing to and reading from a pipe? Faster than POSIX shared memory?

Re: Moving beyond fork() + exec()

#165

Post by trumpdong »

jcranmer wrote: post #144
uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

Calling that elegant is a path dependence of the history of fork+exec. In an alternative world where fork+exec never existed, a lot of those "usual APIs" would probably have had an explicit pid argument to them that let you modify process configuration from a different process. (This is how Fuschia works, e.g.). There's a lot of benefit to this world: the most obvious is that you don't have to magic up some IPC syste…

Or you could call ptrace_syscall (that doesn't currently exist) on your child processes that you are tracing because you'd always be tracing them by default, or get an io_uring for the child process, or...

Re: Moving beyond fork() + exec()

#166

Post by p_l »

pizlonator wrote: post #57
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

Fork is marvelous for the zygote pattern Hard to come up with an optimization that is equally efficient and elegant

And so easy to make into bottleneck.

Yes, zygote pattern makes it easy to make fork() into bottleneck - it requires a lot more discipline and low level tricks (linker scripts, compiler-specific extensions, custom sections, low level dependencies on pagesize that get "fun" on ARM servers).

If you don't, you might wake up with fork() causing latency issues.

Re: Moving beyond fork() + exec()

#167

Post by trumpdong »

garaetjjte wrote: post #63
uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

That's mostly papering over design mistake that most syscalls doesn't accept target pid. Otherwise you could just create suspended process, configure it with syscalls that explicitly take target pid, and start it.

Or have a syscall that runs any other syscall in a different process.

Re: Moving beyond fork() + exec()

#168

Post by trumpdong »

wmf wrote: post #96
ajkjk wrote: post #88

Fork always seemed conceptually terrible even when I first learned about it.. If you want to do one thing (start a process) you should not have to use a mysterious incantation that does a different unrelated thing (forks your process) in order to do it. I am curious about what the best way to handle the example in the article of one process spawning many git subprocesses is. Surely it just doesn't make sense to repea…

libgit2 exists. You could imagine communicating with some gitd over a pipe/socket but I don't know why that would be a good idea. Short of that you have to spawn processes.

On Windows maybe it would be a COM server, using IPC built into the OS. The client sees it like a local function call.

Re: Moving beyond fork() + exec()

#169

Post by dcrazy »

jerf wrote: post #31
sanderjd wrote: post #23

Earlier quoted context omitted.

A process that shares nothing with the process that spawned it.

A thing that makes that complicated is that while you want that conceptually, you don't want that in reality. For instance, if the spawning process is in a container of some sort and it spawned a process that "shares nothing with the process that spawned it", the spawned process would no longer be in that container, because the state of "being in the container" is one of the things it shares with the parent process.…

It’s such a bad idea that every OS except Linux implements it? On macOS it’s posix_spawn, on Windows it’s NtCreateProcess.

Re: Moving beyond fork() + exec()

#170

Post by dcrazy »

JdeBP wrote: post #137
yxhuvud wrote: post #103

Earlier quoted context omitted.

And how do you think posix_spawn is implemented?

This is an oft-overlooked point. An obvious place to look for improving fork+execve is to see whether posix_spawn can be given more efficient kernel mechanisms to be based upon. And of course that has already been done. On NetBSD, posix_spawn() is a fully-fledged system call and much of the work is done in kernel mode. * https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added

This is literally discussed in the article this post links to.
Post reply on HN