Moving beyond fork() + exec() • Hacker News Board

Re: Moving beyond fork() + exec()

#331

Post by kps » Sun, Jun 07, 2026, 4:05 PM UTC

Too wrote: post #279

kps wrote: post #224

Earlier quoted context omitted.

Fork is conceptually simple. Without bringing in any other layers, you start a process with the one thing known to exist: yourself. Otherwise you need multiple steps to create a process, fill it with something to run, and arrange for it to execute. Or like Win32 you permanently smush them together with other layers, like filesystems and object loaders and linkers.

Fill with what stuff exactly? The only thing I want to inherit from the parent process is its cwd and environment variables, even those are often overridden. The rest can easily be passed explicitly through other channels like pipes or command line arguments. Back to the example from the article. It makes no sense that a git-subprocess forked from a web server need to have any process state inherited from the web ser…

> Fill with what stuff exactly?

Yes, exactly. Cloning, as a process creation primitive, is the one thing that doesn't need to be concerned with other stuff.

> … a git-subprocess forked from a web server …

That's pulling in a whole load of assumptions that are distinct from process creation. You can have processes in an environment that has no concept of file system or persistent storage at all.

Re: Moving beyond fork() + exec()

#332

Post by sanderjd » Sun, Jun 07, 2026, 4:11 PM UTC

m132 wrote: post #318

sanderjd wrote: post #264

Earlier quoted context omitted.

I don't know Node or PowerShell very well, but I'm not sure what you mean by this with respect to Python.

CPython doesn't usually create subprocesses unless specifically asked to, it loads Python modules and native extensions into its process. The former is similar (you're still extending an existing process with new code, just interpreted), the latter is literally dlopen(), so loading dynamic libraries. A lot of other Python implementations don't have the ability to spin up new processes at all too.

I still don't really get this point. It's just two different things, spawning processes and running libraries. Seems like you're comparing apples and oranges to me.

Re: Moving beyond fork() + exec()

#333

Post by uecker » Sun, Jun 07, 2026, 4:15 PM UTC

IshKebab wrote: post #330

uecker wrote: post #324

Earlier quoted context omitted.

Define "hack".

Something that works but is a surprising and suboptimal way to do things. I dunno, that's the best I can do for now. Maybe you can do better?

I don't think it is hack. I think it is a nice and clean API and the hate is largely irrational. I think one could improve usability for multi-threaded programs though.

Re: Moving beyond fork() + exec()

#334

Post by pizlonator » Sun, Jun 07, 2026, 4:47 PM UTC

cyberax wrote: post #281

pizlonator wrote: post #57

Earlier quoted context omitted.

Fork is marvelous for the zygote pattern Hard to come up with an optimization that is equally efficient and elegant

Unless you want to create a thread in your zygote. Then it breaks down. Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process.

You can create threads in the zygote. It doesn't "break down", but sure, there's a bit more work.

My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable, preferably lazily (they resume when they are actually needed). So, the zygotes are sitting with those threads suspended. When they become active, they can do work immediately. They might lazily resume those threads as needed.

There are other idioms for this too.

> Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process.

Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives

Re: Moving beyond fork() + exec()

#335

Post by high_byte » Sun, Jun 07, 2026, 4:57 PM UTC

all this for 2%

Re: Moving beyond fork() + exec()

#336

Post by zbentley » Sun, Jun 07, 2026, 6:56 PM UTC

a-dub wrote: post #124

i thought this was all fixed with special modes of clone that are optimized and don't actually copy anything (ie, it creates a new deficient process that can pretty much only exec)?

Kind of. Those exist, but because Linux’s formal ABI is syscalls and not libraries that combine them in known-safe ways, the clone speedups that make fork faster are a confusing and fragile API for low-level programmers to use.

That, and even those clone-without-pagetable-copy improvements leave a lot of slowness on the table. Being able to skip even disable-able functionality intended for fork would simplify code. Also, for programs that launch the same subprocess many times, a better API might allow caching away some of the pre-entrypoint initialization of exec.

Re: Moving beyond fork() + exec()

#337

Post by loeg » Sun, Jun 07, 2026, 7:37 PM UTC

vlovich123 wrote: post #252

loeg wrote: post #156

Earlier quoted context omitted.

In what sense can you not retrofit the zygote pattern?

I recommend at least skimming the paper as it covers this. But essentially you can’t just inject a call at a random point in code to start being a zygote. It’s something you have to plan up front as to the exact point you’re going to fork and that you’re going to do it at the start of program before any threads have started or any files are open and before any locks have been acquired. It’s basically all the challeng…

Sure, but you can always retrofit a program to fork early on... this is a relatively trivial change. No?

Re: Moving beyond fork() + exec()

#338

Post by tosti » Sun, Jun 07, 2026, 8:01 PM UTC

pjmlp wrote: post #149

sunshowers wrote: post #140

Earlier quoted context omitted.

The problem is that threads are not fault boundaries but processes are. So they're not interchangeable when you care about resilience and misbehaving code.

True, but on Windows the approach is then to use COM servers, which have a faster IPC model, and can even serve multiple clients, depending on how the appartement space is configured.

That's like comparing apples and oranges. When tooling is tied to a platform, you're adding in the entire platform to the comparison.

Mozilla implemented an alternative to COM, called XPCOM. XP here means cross platform. Perhaps you could compare against that to take the platform out of the equation.

Re: Moving beyond fork() + exec()

#339

Post by yencabulator » Sun, Jun 07, 2026, 9:15 PM UTC

tliltocatl wrote: post #320

mort96 wrote: post #239

Earlier quoted context omitted.

I would say that pipes and shared memory are the IPC mechanisms? Controlling the state of the exec'd process's file descriptors would counts as a way to set up interprocess communication, but once that's done, it's the pipe or SHM that does the actual communication.

The problem with POSIX IPC is that passing file descriptors between processes (other than parent passing to child via fork) is hard. Yes, SCM_RIGHTS can do it, but it is quite error prone and rarely done.

Every single Wayland and GPU-accelerated X11 app does that all the time.

Re: Moving beyond fork() + exec()

#340

Post by dwattttt » Sun, Jun 07, 2026, 9:16 PM UTC

uecker wrote: post #317

dwattttt wrote: post #313

Earlier quoted context omitted.

Doing the same thing on Windows, I create the pipe and get two ends in the same process. Then I'd call CreateProcess and indicate I want the pipe's handle (fd) inherited to the child, and I'd use a prearranged way to tell the child what the fd value is it should use. Possibly the most common way to tell the child the value is by setting it as a CLI arg in CreateProcess.

Yes, and CreateProcess needs special facilities to make this possible while in the UNIX model you don't.

Which special facilities are you referring to? If it's the ability to selectively inherit handles (fds) to a new process, Linux's lack of this "special facility" is nothing to be proud of.

How do you selectively pass on fds without having a global impact on your process?