Moving beyond fork() + exec() • Hacker News Board

Re: Moving beyond fork() + exec()

#181

Post by plorkyeran » Sat, Jun 06, 2026, 7:08 PM UTC

Someone wrote: post #159

nvme0n1p1 wrote: post #106

Earlier quoted context omitted.

We don't have any broadly used non-fork samples. Windows, macOS, and Linux all have fork. So the presence of fork can't be the reason for the performance difference. (Windows's fork is called ZwCreateProcess)

MacOS has posix_spawn . See https://developer.apple.com/library/archive/documentation/Sy... (yes, that’s an iOS man page. MacOS has the call, too, but I couldn’t find the man page online and it looks identical to me) I don’t know how they implemented it, though. Under the hood, it could do the equivalent of a fork / exec pair.

XNU's posix_spawn implementation is not fork/exec-based. It does roughly what the API suggests it would do.

Re: Moving beyond fork() + exec()

#182

Post by asveikau » Sat, Jun 06, 2026, 7:08 PM UTC

loeg wrote: post #172

asveikau wrote: post #135

The things you can do between fork and exec are sometimes underestimated. Off the top of my head, you can call dup2(), you can set a process group id, probably a few other things. If you contrast that with win32, where you optionally pack a bunch of initial values into a struct, win32 is a much more narrow, less pleasant, less freeform interface, where it is harder to introduce more features. But I think there is alr…

> The things you can do between fork and exec are sometimes underestimated. Off the top of my head, you can call dup2(), you can set a process group id, probably a few other things. What do you mean underestimated? You can do anything between fork and exec; there are no limitations.

That's not true. Just one example, if you do anything with threads you are pretty screwed. For example if another thread holds a mutex at the time of fork(2), and you also want that mutex.

Re: Moving beyond fork() + exec()

#183

Post by JdeBP » Sat, Jun 06, 2026, 7:10 PM UTC

dcrazy wrote: post #170

JdeBP wrote: post #137

Earlier quoted context omitted.

This is an oft-overlooked point. An obvious place to look for improving fork+execve is to see whether posix_spawn can be given more efficient kernel mechanisms to be based upon. And of course that has already been done. On NetBSD, posix_spawn() is a fully-fledged system call and much of the work is done in kernel mode. * https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added

This is literally discussed in the article this post links to.

Not really. They didn't get anywhere near as far as noticing the prior art of NetBSD, not even on the mailing list discussion behind that article.

Re: Moving beyond fork() + exec()

#184

Post by asveikau » Sat, Jun 06, 2026, 7:10 PM UTC

dcrazy wrote: post #173

asveikau wrote: post #135

The things you can do between fork and exec are sometimes underestimated. Off the top of my head, you can call dup2(), you can set a process group id, probably a few other things. If you contrast that with win32, where you optionally pack a bunch of initial values into a struct, win32 is a much more narrow, less pleasant, less freeform interface, where it is harder to introduce more features. But I think there is alr…

posix_spawn is emulated on Linux, but it is a native syscall on macOS (and possibly other OSes?). As discussed in the linked article, there is interest in changing Linux to adopt this model, where posix_spawn is its own fundamental primitive.

Yeah, I think it is a reasonable transition path or implementation detail for some systems to implement it in userland atop fork(2), and others to natively spawn a new process without copying the old address space.

Re: Moving beyond fork() + exec()

#185

Post by monocasa » Sat, Jun 06, 2026, 7:21 PM UTC

201984 wrote: post #160

monocasa wrote: post #98

Earlier quoted context omitted.

That's not really patching though, any more than any use of function pointers is patching.

There's a part of the .so ELF file (the Global Offset Table aka GOT) that has to be modified with all the addresses of the functions being imported, which of course vary from process to process. If not patching, what exactly would you call modifying part of the file?

And the got is just a big table of pointers like any other table of pointers your application manipulates as it runs.

This isn't meant as a reductive take, but instead that there is a difference between completely describable in C like the contents of the .got section, and something like a .reloc section that actually has to understand the generated assembly in order to build the relocation table to load and link the executable. Both are linking, but I've saved "patching" for more brain surgery esque techniques. Like on mips, the jump instruction immediate is the bottom 26 bits of the absolute address of the target, so you're going through and modifying all of the jump instructions if you load it to somewhere it wasn't linked at.

Re: Moving beyond fork() + exec()

#186

Post by josefx » Sat, Jun 06, 2026, 7:24 PM UTC

cls59 wrote: post #69

mrkeen wrote: post #35

> fork() is a relatively expensive system call; it must copy the entire process state (including memory) for the child process. Many optimizations have been made over the years, but a fork is still a fundamentally costly operation. To make things worse, a fork() call is often immediately followed by an exec(), which will discard all of that memory that was so carefully copied for the child. It's weird to leave out a…

Even with copy-on-write, fork() still has to pay the setup cost for COW. If the parent process has a lot of busy threads (e.g. Java), you can end up doing a lot of unnecessary COW before exec() fires.

Isn't that what vfork tried to address? No COW, the child starts in its parents address space and only gets its own after calling exec.

Re: Moving beyond fork() + exec()

#187

Post by JdeBP » Sat, Jun 06, 2026, 7:33 PM UTC

PaulDavisThe1st wrote: post #116

pjmlp wrote: post #70

Earlier quoted context omitted.

Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.

A more accurate way to describe this is that Windows' (NT onward) core execution context model is a bunch of threads that by default share memory, whereas Unixen have a core task context model of a bunch of threads that by default do not share memory. Both systems are implemented using threads as the execution context, but in Unix, the history means that that you fork+exec most of the time, resulting in a two tasks t…

That's actually less accurate, not more. It's a post-hoc revision that conflates Unix with Linux.

The Unix model was invented over a decade before the idea of multithreading percolated into mainstream operating systems at all.

The reason that Windows NT started as it did, was that OS/2 had come out in 1987, with kernel threads, and the idea of multithreading had taken root. SunOS 5 gained threading, too.

Windows NT applications development began with threading available as a mechanism from the start, and with a lot of people in the IBM/Microsoft world already knowing about its use in applications development from OS/2.

Whereas with the Unices it came in more gradually, as the applications had often already been designed. The whole libthread versus libpthread thing made things interesting on SunOS for a few years, too. As did the first attempt (LinuxThreads) at providing threads on Linux.

Re: Moving beyond fork() + exec()

#188

Post by not_a_bijection » Sat, Jun 06, 2026, 7:34 PM UTC

Animats wrote: post #123

rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returne…

I think fork() is more of a PDP-7 mistake than a PDP-11 mistake. On the original UNIX system, memory was so limited that the only sane partitioning was to write the running program's memory image to disk, then reuse the running image as the child. An immediate consequence is the UNIX I/O model, where disk I/O is always synchronous (can't swap processes while waiting for disk I/O because swapping processes requires disk I/O). Anyway, as soon as the UNIX group got a PDP-11, the model broke down, because they had enough memory for multiple processes, but fork() didn't allow them to run concurrently, because their first PDP-11 didn't have an MMU. So they whined until they got one with an MMU instead of fixing their broken design.

Re: Moving beyond fork() + exec()

#189

Post by skydhash » Sat, Jun 06, 2026, 7:37 PM UTC

tadfisher wrote: post #177

skydhash wrote: post #143

Earlier quoted context omitted.

Does it need to be the same OS? Most consumer device are in the low 16GB range for memory with some outliers in the 64 and 128 GB. 32 cores are still in the realm of specialized devices. Yes, we’re not the one paying for Linux development, but its subsystems are so complicated for general purpose computing. Like fitting formula 1 car parts onto a camry.

Our software is littered with the consequences of these kinds of assumptions, and they have an impact on consumer use cases. x86 still runs in real mode on boot despite dropping the PC BIOS. Lots of software still assumes a 4kb page size, to the point where migrating Android to 16kb is an ongoing multi-year effort involving far too many people. And this is an OS for phones, which you might assume would lack the memor…

It’s all about contracts. It’s fine to define assumptions and build software on top of those. It’s also fine to break those and adjust the software. The trap is trying to steer towards a universal solution (Yagni is the cure there) or trying to slip something in that does not respect the contracts (hence bugs).

UEFI could have supported something like ELF and do away with real mode. Intel and Amd could have just introduced a new line of cpu and everyone could have transitioned to that (with maybe shims to soften the change). But everyone is all about backwards compatibility and compile once, runs for eternity.

Re: Moving beyond fork() + exec()

#190

Post by duped » Sat, Jun 06, 2026, 7:39 PM UTC

Animats wrote: post #123

rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returne…

> It links to a .so file which reads the executable header, allocates memory, loads the program, gets it ready to run, and starts it. The program loader runs in user space and is unprivileged. This is probably the right way to do it.

aiui this is what exec does, the problem outlined here is the split between process creation (expensive, kernel space, has to be done each time even if spawning the same process "template" repeatedly) and loading (cheap and in userspace).