Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

141–150 of 358 posts

Re: Moving beyond fork() + exec()

#141

Post by codedokode »

The problem with replacing exec/fork is that you usually want to configure new process: for example, set up signal handlers, close or open FDs, switch namespaces, setup seccomp, adjust permissions. And all the system calls to do it apply only to the current process and you need something to replace them. The proposal in the article was to create a new API for this.

My idea is that we could make a new syscall, for example "spawn", that creates a new empty process, loads some lightweight "loader" into it, and passes arbitrary configuration data. The loader configures the process and exec()'s the main program. This allows to avoid forking the memory and keep existing APIs, but still requires to fork file descriptors and other things.

Re: Moving beyond fork() + exec()

#142

Post by theK »

mort96 wrote: post #112
theK wrote: post #94

Earlier quoted context omitted.

Didn't he just say that fork turns out to be comparatively faster to the non-fork samples we get? Ie Linux spawns processes faster than Microsoft's kernels?

Didn't I just say that "the problem with fork isn't really that it's slow"? It's all the other OS design choices it forces on you if you want it to be fast.

Right, you did. I somehow misread your comment.

Re: Moving beyond fork() + exec()

#143

Post by skydhash »

redleader55 wrote: post #127
ggm wrote: post #118

Aesthetically I have no intention of moving beyond. I'm content with my kernels scheduler and how it maps "heavyweight" processes to cores. I do use threaded code. It's significantly harder to write and reason about. (45 years in to a CS career, ageing out) You have to be clever to do better than clever people. Clever people bootstrapped me into fork()/exec() and I know my limits.

When cores start needing more than 9 bits to be represented and RAM is in terabytes, many of the old assumptions need to change. Schedulers need to be implemented in userspace, RAM needs to be allocated in GB, not in 4k, io needs to require less round-trips between kernel and user space and NICs need to do a lot more work before the data reaches the CPU.

Does it need to be the same OS? Most consumer device are in the low 16GB range for memory with some outliers in the 64 and 128 GB. 32 cores are still in the realm of specialized devices.

Yes, we’re not the one paying for Linux development, but its subsystems are so complicated for general purpose computing. Like fitting formula 1 car parts onto a camry.

Re: Moving beyond fork() + exec()

#144

Post by jcranmer »

uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

Calling that elegant is a path dependence of the history of fork+exec.

In an alternative world where fork+exec never existed, a lot of those "usual APIs" would probably have had an explicit pid argument to them that let you modify process configuration from a different process. (This is how Fuschia works, e.g.). There's a lot of benefit to this world: the most obvious is that you don't have to magic up some IPC system just to report configuration errors, but there's actually a good amount of utility in being able to have a manager process that is tweaking attributes of its children (e.g., debuggers would love it).

Re: Moving beyond fork() + exec()

#145

Post by pjmlp »

snozolli wrote: post #138
PaulDavisThe1st wrote: post #116

Earlier quoted context omitted.

A more accurate way to describe this is that Windows' (NT onward) core execution context model is a bunch of threads that by default share memory, whereas Unixen have a core task context model of a bunch of threads that by default do not share memory. Both systems are implemented using threads as the execution context, but in Unix, the history means that that you fork+exec most of the time, resulting in a two tasks t…

whereas Unixen have a core task context model of a bunch of threads that by default do not share memory. How are those not simply child processes? I don't understand your use of the word 'threads' here. Does the Unix world not distinguish between threads and processes? In Win32, threads exist within processes, and you can create new threads or child processes.

Actually on Windows a process is a thread with additional information.

The unit of execution is the thread.

On the UNIX world it depends on which UNIX you are talking about.

Linux has a similar model to Windows NT nowadays, hence clone() as key primitive.

Other UNIXes have different approaches.

Re: Moving beyond fork() + exec()

#146

Post by nyrikki »

codedokode wrote: post #141

The problem with replacing exec/fork is that you usually want to configure new process: for example, set up signal handlers, close or open FDs, switch namespaces, setup seccomp, adjust permissions. And all the system calls to do it apply only to the current process and you need something to replace them. The proposal in the article was to create a new API for this. My idea is that we could make a new syscall, for exa…

Luckily someone with a time machine saw your post and added it to POSIX.1-2001 :)

(Sorry if you weren't joking) but yes, posix_spawn() has been a thing and in glibc fork is just a alias to clone()

Not exactly that OP idea, but fork/exec is legacy really.

Re: Moving beyond fork() + exec()

#147

Post by trumpdong »

I liked the other proposal where you can create a blank process and then force it to make syscalls, ending with execve. That doesn't require a bunch of special data structures to hold the syscalls you want to do.

Re: Moving beyond fork() + exec()

#148

Post by dcrazy »

Animats wrote: post #123
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returne…

Don’t pretty much all OSes implement process startup in userspace? On macOS, the kernel creates a process with an image of dyld and points it at dyld_start, which actually takes care of parsing the Mach-O header. I assumed ld.so does the same job on Linux.

Re: Moving beyond fork() + exec()

#149

Post by pjmlp »

sunshowers wrote: post #140
pjmlp wrote: post #70

Earlier quoted context omitted.

Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.

The problem is that threads are not fault boundaries but processes are. So they're not interchangeable when you care about resilience and misbehaving code.

True, but on Windows the approach is then to use COM servers, which have a faster IPC model, and can even serve multiple clients, depending on how the appartement space is configured.

Re: Moving beyond fork() + exec()

#150

Post by trumpdong »

snozolli wrote: post #138
PaulDavisThe1st wrote: post #116

Earlier quoted context omitted.

A more accurate way to describe this is that Windows' (NT onward) core execution context model is a bunch of threads that by default share memory, whereas Unixen have a core task context model of a bunch of threads that by default do not share memory. Both systems are implemented using threads as the execution context, but in Unix, the history means that that you fork+exec most of the time, resulting in a two tasks t…

whereas Unixen have a core task context model of a bunch of threads that by default do not share memory. How are those not simply child processes? I don't understand your use of the word 'threads' here. Does the Unix world not distinguish between threads and processes? In Win32, threads exist within processes, and you can create new threads or child processes.

They are child processes.

Second answer: Linux doesn't differentiate between threads and processes. It has a "thread group ID" that serves a small number of purposes, and the rest of the difference is just whether the threads happen to share the same address space.

Post reply on HN