Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

31–40 of 358 posts

Re: Moving beyond fork() + exec()

#31

Post by jerf »

sanderjd wrote: post #23
dnw wrote: post #20

Earlier quoted context omitted.

What do you mean by "a completely new process"?

A process that shares nothing with the process that spawned it.

A thing that makes that complicated is that while you want that conceptually, you don't want that in reality. For instance, if the spawning process is in a container of some sort and it spawned a process that "shares nothing with the process that spawned it", the spawned process would no longer be in that container, because the state of "being in the container" is one of the things it shares with the parent process.

This is just an example of I don't even know how many things a modern-day process will share from its parent.

By "complicated" I do not even remotely mean "unsolvable". I just mean that if you really dig down into what it means to "share nothing" in a modern operating system, it's a lot richer than it was back when fork+exec was a practical solution. There's a lot of fuzzy things that could go either way when you say "shares nothing".

Re: Moving beyond fork() + exec()

#32

Post by ptspts »

sjmulder wrote: post #28
201984 wrote: post #4

Earlier quoted context omitted.

Shared libraries (and mmapped files in general) are deduplicated; it's nowhere near as bad as you think. The kernel loads a .so into memory once and then maps that memory into every process that mmaps it. Editing to add: this deduplication is one of the greatest upsides to dynamic linking. Common libs like libgcc and libc only have to exist in memory once and can stay in CPU caches, whereas if they were statically li…

Doesn't the loaded code have to be patched for relocations?

It does, so not 100% is reused. The patched parts are in different sections though, so the entire .text (code) section ends up being reused.

Re: Moving beyond fork() + exec()

#33

Post by wongarsu »

dnw wrote: post #20
sanderjd wrote: post #17

I just ran into this recently, where I had an obscure bug caused by needing to close more file descriptors in the forked process. "I want a clone of the current process" is just way less common in my experience than "I want a completely new process". It feels crazy that we don't have a way to directly express the latter thing, and can only approximate it by cloning and then fixing things up in post.

What do you mean by "a completely new process"?

The equivalent of CreateProcessW https://learn.microsoft.com/en-us/windows/win32/api/processt...

Re: Moving beyond fork() + exec()

#34

Post by monocasa »

sjmulder wrote: post #28
201984 wrote: post #4

Earlier quoted context omitted.

Shared libraries (and mmapped files in general) are deduplicated; it's nowhere near as bad as you think. The kernel loads a .so into memory once and then maps that memory into every process that mmaps it. Editing to add: this deduplication is one of the greatest upsides to dynamic linking. Common libs like libgcc and libc only have to exist in memory once and can stay in CPU caches, whereas if they were statically li…

Doesn't the loaded code have to be patched for relocations?

Not on modern archs that provide decent support for PIE (position independent executables).

Re: Moving beyond fork() + exec()

#35

Post by mrkeen »

> fork() is a relatively expensive system call; it must copy the entire process state (including memory) for the child process. Many optimizations have been made over the years, but a fork is still a fundamentally costly operation. To make things worse, a fork() call is often immediately followed by an exec(), which will discard all of that memory that was so carefully copied for the child.

It's weird to leave out a mention of copy-on-write - the optimisation that means that you don't copy over all the memory.

Re: Moving beyond fork() + exec()

#36

Post by rom1v »

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/...

> ABSTRACT

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog the ways in which fork is a terrible abstraction for the modern programmer to use, describe how it compromises OS implementations, and propose alternatives.

> As the designers and implementers of operating systems, we should acknowledge that fork’s continued existence as a first-class OS primitive holds back systems research, and deprecate it. As educators, we should teach fork as a historical artifact, and not the first process creation mechanism students encounter.

Re: Moving beyond fork() + exec()

#37

Post by lanstin »

dijit wrote: post #14
Sophira wrote: post #8

I'm guessing that a big part of the problem with moving away from fork() in general is that each new process needs a copy of the parent process' environment anyway, right?

I'm a bit naive, but I don't think that's necessarily a requirement. It might be commonly held convention, and thus, an assumption, in Linux (and, broadly, UNIX) but I don't think it's true inside VAX or even Windows, so I don't think it's a requirement . Unless I've missed something (which is totally possible, this is not an area of OS design I've spent much time).

But also UID, groups, controlling TTY, process group, capabilities, pipes, shared memory, etc. and the file descriptors while maybe not inherently needed are how a lot of Unix plumbing works.

Re: Moving beyond fork() + exec()

#38

Post by t-3 »

sjmulder wrote: post #28
201984 wrote: post #4

Earlier quoted context omitted.

Shared libraries (and mmapped files in general) are deduplicated; it's nowhere near as bad as you think. The kernel loads a .so into memory once and then maps that memory into every process that mmaps it. Editing to add: this deduplication is one of the greatest upsides to dynamic linking. Common libs like libgcc and libc only have to exist in memory once and can stay in CPU caches, whereas if they were statically li…

Doesn't the loaded code have to be patched for relocations?

Not if it's position-independent.

Re: Moving beyond fork() + exec()

#39

Post by 1718627440 »

sanderjd wrote: post #17

I just ran into this recently, where I had an obscure bug caused by needing to close more file descriptors in the forked process. "I want a clone of the current process" is just way less common in my experience than "I want a completely new process". It feels crazy that we don't have a way to directly express the latter thing, and can only approximate it by cloning and then fixing things up in post.

But you generally want to communicate with that process, so you do need to setup e.g. file descriptors and stuff, which needs information from the parent process to be passed.

Re: Moving beyond fork() + exec()

#40

Post by burnt-resistor »

> "If you are repeatedly creating large processes, you are already doing it wrong. The fix is in user space, not the kernel."

Every couple of years, someone claims they have "the solution" implying everyone else who came before them didn't know what they were doing.

Post reply on HN