Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

71–80 of 358 posts

Re: Moving beyond fork() + exec()

#71

Post by [deleted] »

anarazel wrote: post #58
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

[deleted]

Re: Moving beyond fork() + exec()

#72

Post by nvme0n1p1 »

anarazel wrote: post #58
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

That's not the reason for the performance difference. Windows does have a fork primitive (ZwCreateProcess) and it's still slower than Linux's equivalent.

Re: Moving beyond fork() + exec()

#73

Post by jcalvinowens »

It is a weirdly common misconception that that fork() is cheap... it is O(N) on the size of the process, and it always has been.

Yes, it's copy on write... but there is a linear relationship between the size of the process and the number of page table entries required to represent it.

Re: Moving beyond fork() + exec()

#74

Post by toast0 »

pizlonator wrote: post #57
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

Fork is marvelous for the zygote pattern Hard to come up with an optimization that is equally efficient and elegant

The zygote pattern[1] is a great optimization to deal with the cost of forking, but IMHO, being able to inexpensively spawn a carefully tailored process regardless of the size and scope of the current process would be better.

I would guess it would be a small difference in measurable performance between zygote and a direct clean spawn, but it's one less trick an application needs to do, and it would be very helpful for libraries that spawn things. Spawning inside a library isn't always a great thing to do, but some things would really benefit from process level isolation.

[1] In case one isn't aware, the zygote pattern involves forking a 'zygote' process during application startup, and having that process do any forks that need to happen during application runtime. This reduces the cost of forking in large applications, because the zygote will have few fds open and use little memory. This lets your large application spawn new processes without delaying the application or the startup of the new processes. Some applications will spawn many zygotes to allow parallelism for spawning at runtime.

Re: Moving beyond fork() + exec()

#75

Post by epcoa »

mrkeen wrote: post #35

> fork() is a relatively expensive system call; it must copy the entire process state (including memory) for the child process. Many optimizations have been made over the years, but a fork is still a fundamentally costly operation. To make things worse, a fork() call is often immediately followed by an exec(), which will discard all of that memory that was so carefully copied for the child. It's weird to leave out a…

> It's weird to leave out a mention of copy-on-write

For the intended audience of such a paper this is base knowledge.

Re: Moving beyond fork() + exec()

#76

Post by aseipp »

anarazel wrote: post #58
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

I suspect it's a long tail sort of thing; it mostly doesn't matter except when it really matters. It's interesting that the stated motivation for the patch is in the context of agentic tools spawning subcommands. There's some related prior art in this area where the payoffs could be much greater, like fuzzing: https://gts3.org/assets/papers/2017/xu:os-fuzz.pdf is an example. It would be very interesting to see this patch applied to e.g. AFL++

Re: Moving beyond fork() + exec()

#77

Post by __david__ »

uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

I agree. I think the current way is very nice to use (in c). I think the best way would be to have something similar to vfork() but not bound by posix rules. Then make the normal posix apis (close, setuid, etc.) act like the Rust “builder” pattern. Possibly giving them a prefix for explicitness. That way the “fill out a giant structure” people could have their wish and the people that just want a faster posix experience don’t have to learn an entirely new concept and api surface. It would be future extensible that way, too (just add more prefixed calls to the builder).

Re: Moving beyond fork() + exec()

#78

Post by zozbot234 »

pjmlp wrote: post #70
anarazel wrote: post #58

Earlier quoted context omitted.

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.

Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.

Re: Moving beyond fork() + exec()

#79

Post by mort96 »

anarazel wrote: post #58
rom1v wrote: post #36

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog t…

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

The problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1, and you need overcommit.

Now these decisions aren't objectively bad, but they have significant trade-offs and it's probably not a good idea that they're forced simply because we use fork()+exec() for process creation.

Re: Moving beyond fork() + exec()

#80

Post by wvenable »

burnt-resistor wrote: post #47
amluto wrote: post #29

Earlier quoted context omitted.

I have the entirely opposite opinion. IMO a big mistake of the UNIXy model is that so much state is preserved across the creation of a process. For example, there are APIs to have a specific thing be fd number 4 so you can run a program and have it find that thing at fd 4. This is weird . Windows, for all its many, many faults, did not use fork+exec and instead mostly has options for how one creates a process. It was…

You're simply failing to grasp the value of the simplicity, compatibility, and portability of POSIX/*nix. Inventing yet another way to create a process would be complex and break things. It's a-la-carte to enable configuration after fork of the new CoW or non-CoW process but before exec (unless vfork or similar were used). This is the model. If you want to greenfield re-engineer the world with all new system calls an…

"The reasonable man adapts himself to POSIX: the unreasonable one persists in trying to adapt the POSIX to himself. Therefore all progress depends on the unreasonable man."

― George Bernard Shaw, probably.

Post reply on HN