Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A fork() in the road

microsoft.com

Post reply on HN
Print view View original

71–80 of 184 posts

Re: A fork() in the road

#71

Post by alkonaut »

bloak wrote: post #63
alkonaut wrote: post #57

Earlier quoted context omitted.

I can’t answer for Nginx but normally on windows if you want “worker processes” you just start N of them and have them read work from a shared memory queue. That is, workers live longer than the tasks they perform. If one crashes, a new one is spawned. This does seem like a more sensible way of doing things than forking tbh. It isolates work in processes but doesn’t pay for process creation per request.

"If one crashes, a new one is spawned." I suppose that makes sense on an OS on which crashing is expected behaviour, though some people would want to know what bug caused the crash and whether that bug has security implications.

There is, of course, absolutely no difference in how errors are handled in this case vs forks. A process that handles one million requests is much more likely to crash than a process that handles one request though (regardless of OS).

Re: A fork() in the road

#72

Post by dijit »

alkonaut wrote: post #55
localhostdotdev wrote: post #53

"every other system has a feature except us, and we are not going to add it (because reasons) even if it's very widely used" also: > When a fork syscall is made on WSL, lxss.sys does some of the initial work to prepare for copying the process. It then calls internal NT APIs to create the process with the correct semantics and create a thread in the process with an identical register context. Finally, it does some add…

They must have considered it many times (not least when making the partial posix support for NT) but felt that supporting it wouldn’t help deprecating it either. AFAIK it’s only unix/Linux (posix) OSes that implement fork. Perhaps that’s what you meant by “every other system”, ie unix + clones/derivatives?

VAX/VMS implement "vfork()" which is an implementation of the most common use-cases of fork.

VAX and VMS are not POSIX or UNIX-like.

Re: A fork() in the road

#73

Post by fanf2 »

speedplane wrote: post #35
sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

I used to work on a cross-platform project, and spent several weeks trying to figure out why our application ran significantly faster on linux than windows. One major culprit was process creation (another was file creation). I never really uncovered the true reason, but I suspect it had to do with the large number of DLLs that Windows would automatically link if you weren't very careful. Linux, of course, can also lo…

Anti-virus software makes process and file operations a lot slower.

Re: A fork() in the road

#74

Post by kazinator »

afiori wrote: post #62
kazinator wrote: post #54

Fork is quite excellent, except in cases when the intent is to run a different program or when threads are involved (threads are basically an incompatible, competing model of concurrency). The use of fork as a concurrency mechanism (creating a new thread of control that executes in a copy of the address space) is very good and useful. In the POSIX shell language, the subshell syntax (command1; command2; ...) is easil…

The paper agrees with you that the fork models had a reason to exist and that is is perfect for shells. They also point out that on modern hardware you often should want to write multithreaded multiprocess application. Their main criticism of fork is that it does not compose at any level of the OS (as it cannot be implemented over a different primitive) I understand that a lot of people here dislike Microsoft for goo…

It's an idiotic argument. Only functions compose. Though fork is packaged as a function, it's really an operator with a big effect.

Booting a system doesn't compose; let's not have power-on reset and bootloaders.

Everything in this paper could have been cribbed from twenty year or older Usenet postings, mailing lists and other sources. Fork has been dissected ad nausem; anyone who is anyone in the Unix-like world knows this.

Oh, and threads have perpetually been the way to go on current hardware --- every damn year since 1988 and counting.

Re: A fork() in the road

#75

Post by kazinator »

sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

CreateProcess requires an application to initialize from scratch. When you fork, you cheaply inherit the initialized state of the whole application image. Only a few pages that are mutated have to be subject to copy-on-write. Even that copy-on-write is cheaper than calculating the contents of those pages from scratch.

Re: A fork() in the road

#76

Post by chungleong »

richardwhiuk wrote: post #45
muststopmyths wrote: post #30

Earlier quoted context omitted.

If I had to guess, I'd point to DLLs. The minimal Windows process loads probably half a dozen, plus the entry points are called in a serialized manner.

Pretty much identical to shared objects on Linux

Windows DLLs require fixups when they're loaded off their preferred base address.

Re: A fork() in the road

#77

Post by msla »

dijit wrote: post #72
alkonaut wrote: post #55

Earlier quoted context omitted.

They must have considered it many times (not least when making the partial posix support for NT) but felt that supporting it wouldn’t help deprecating it either. AFAIK it’s only unix/Linux (posix) OSes that implement fork. Perhaps that’s what you meant by “every other system”, ie unix + clones/derivatives?

VAX/VMS implement "vfork()" which is an implementation of the most common use-cases of fork. VAX and VMS are not POSIX or UNIX-like.

VAX is a hardware architecture. I'm not one to nitpick, but differentiating between VAX and VMS when VMS ran on the VAX is confusing.

Re: A fork() in the road

#79

Post by JdeBP »

acqq wrote: post #38
Dylan16807 wrote: post #27

Earlier quoted context omitted.

Well that's a complicated question to answer. You can use the posix_spawn function in glibc, which uses a vfork or clone syscall just like on Linux.

Also relevant, regarding the Linux native performance: https://mobile.twitter.com/RichFelker/status/602313979894038... "Rich Felker, May 24, 2015: Some interesting preliminary timing of @musllibc 's posix_spawn vs fork+exec shows it ~25x faster for large parent processes. (~360us vs 9ms). #glibc has a vfork-based posix_spawn but it's only usable for trivial cases; others use fork. @musllibc posix_spawn always uses CL…

See also https://news.ycombinator.com/item?id=18073906 .

Re: A fork() in the road

#80

Post by Gibbon1 »

ralish wrote: post #21
sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

I'm a bit rusty on this but from memory the overhead is by and large specific to the Win32 environment. Creating a "raw" process is cheap and fast (as you'd reasonably expect), but there's a lot of additional initialisation that needs to occur for a "fully-fledged" Win32 process before it can start executing. Beyond the raw Process and Thread kernel objects, which are represented by EPROCESS + KPROCESS and ETHREAD +…

> created on-demand with the first creation of a GDI or USER object, so presumably CLI apps don't have to pay that price

This tickles my brain. I read some blog post bitching that because Windows DLL's are kinda heavy weight it's way easy end up paying that price without realizing it.

Post reply on HN