Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

91–100 of 358 posts

Re: Moving beyond fork() + exec()

#91

Post by JdeBP »

zozbot234 wrote: post #78
pjmlp wrote: post #70

Earlier quoted context omitted.

Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.

Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.

Windows NT was never designed with pre-386 machines in mind. That was the territory of the old DOS+Windows. Windows NT from the get-go was for machines with page-based virtual memory.

* https://computernewb.com/~lily/files/Documents/NTDesignWorkb...

Re: Moving beyond fork() + exec()

#92

Post by chasil »

amluto wrote: post #29
uecker wrote: post #26

The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.

I have the entirely opposite opinion. IMO a big mistake of the UNIXy model is that so much state is preserved across the creation of a process. For example, there are APIs to have a specific thing be fd number 4 so you can run a program and have it find that thing at fd 4. This is weird . Windows, for all its many, many faults, did not use fork+exec and instead mostly has options for how one creates a process. It was…

Well, Cygwin and Busybox have shown me that fork-heavy activities are about 100x slower on Windows than Linux.

The Windows approach may be correct, but it suffers in performance from the POSIX perspective.

I have heard that WSL1 iimproves this.

Re: Moving beyond fork() + exec()

#93

Post by epcoa »

zozbot234 wrote: post #78
pjmlp wrote: post #70

Earlier quoted context omitted.

Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.

Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.

This is not true. NT never had fork, was always based on the assumption of an MMU and Dave Cutler was a well known fork hater in the 80s long before this paper came out and made it cool to be so. By the time Windows 95 was out, the baseline was 386 with an MMU. CreateThread was initially designed for NT in 1993 though (which didn’t support pre-386 CPUs).

Re: Moving beyond fork() + exec()

#94

Post by theK »

mort96 wrote: post #79
anarazel wrote: post #58

Earlier quoted context omitted.

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

The problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1, and you need overcommit. Now these decisions aren't objectively bad , but they have significant trade-offs and it's probably not a go…

Didn't he just say that fork turns out to be comparatively faster to the non-fork samples we get? Ie Linux spawns processes faster than Microsoft's kernels?

Re: Moving beyond fork() + exec()

#95

Post by JdeBP »

anarazel wrote: post #62
stabbles wrote: post #53

Earlier quoted context omitted.

Isn't that covered by O_CLOEXEC?

There's a bunch of nastiness around that too. If you have e.g. library state that assumes the fd still works you can get her very confusing bugs once another file is opened into that fd number...

You may be mixing up fork and exec. Library data state isn't retained over execve(), and O_CLOEXEC does not take effect at fork().

Re: Moving beyond fork() + exec()

#96

Post by wmf »

ajkjk wrote: post #88

Fork always seemed conceptually terrible even when I first learned about it.. If you want to do one thing (start a process) you should not have to use a mysterious incantation that does a different unrelated thing (forks your process) in order to do it. I am curious about what the best way to handle the example in the article of one process spawning many git subprocesses is. Surely it just doesn't make sense to repea…

libgit2 exists. You could imagine communicating with some gitd over a pipe/socket but I don't know why that would be a good idea. Short of that you have to spawn processes.

Re: Moving beyond fork() + exec()

#97

Post by stefan_ »

1718627440 wrote: post #39
sanderjd wrote: post #17

I just ran into this recently, where I had an obscure bug caused by needing to close more file descriptors in the forked process. "I want a clone of the current process" is just way less common in my experience than "I want a completely new process". It feels crazy that we don't have a way to directly express the latter thing, and can only approximate it by cloning and then fixing things up in post.

But you generally want to communicate with that process, so you do need to setup e.g. file descriptors and stuff, which needs information from the parent process to be passed.

Keep in mind that this is the only way to start any process. Even if you just want to launch some throwaway utility program.

Re: Moving beyond fork() + exec()

#98

Post by monocasa »

201984 wrote: post #85
monocasa wrote: post #34

Earlier quoted context omitted.

Not on modern archs that provide decent support for PIE (position independent executables).

How do you think position independent code can call functions from other .so's without being patched with their addresses? They can't, so even PIC code still has to have a relocation table that gets patched. It's in a different page than the code though, so code does still get reused.

That's not really patching though, any more than any use of function pointers is patching.

Re: Moving beyond fork() + exec()

#99

Post by amluto »

chasil wrote: post #92
amluto wrote: post #29

Earlier quoted context omitted.

I have the entirely opposite opinion. IMO a big mistake of the UNIXy model is that so much state is preserved across the creation of a process. For example, there are APIs to have a specific thing be fd number 4 so you can run a program and have it find that thing at fd 4. This is weird . Windows, for all its many, many faults, did not use fork+exec and instead mostly has options for how one creates a process. It was…

Well, Cygwin and Busybox have shown me that fork-heavy activities are about 100x slower on Windows than Linux. The Windows approach may be correct, but it suffers in performance from the POSIX perspective. I have heard that WSL1 iimproves this.

Linux has worked pretty hard to optimize fork(). This doesn’t mean that fork() is a good idea.

Windows does not historically depend on fork(), so there was no native fork(), so Cygwin kludged it up.

Re: Moving beyond fork() + exec()

#100

Post by JdeBP »

__david__ wrote: post #83
amluto wrote: post #29

Earlier quoted context omitted.

I have the entirely opposite opinion. IMO a big mistake of the UNIXy model is that so much state is preserved across the creation of a process. For example, there are APIs to have a specific thing be fd number 4 so you can run a program and have it find that thing at fd 4. This is weird . Windows, for all its many, many faults, did not use fork+exec and instead mostly has options for how one creates a process. It was…

Having fd 4 mean something specific is no weirder than having fds 0,1, and 2 mean something specific, which is probably never going to change. At some point you just gotta embrace the Unix.

Heh! The Unix didn't embrace the idea of file descriptor 3 meaning something specific. (-:

* https://jdebp.uk/FGA/bernstein-on-ttys/cttys.html

Interestingly, on MS/PC/DR-DOS file descriptor 3 was stdaux. and file descriptor 4 was stdprn.

Post reply on HN