Live data from Hacker News

A fork() in the road

microsoft.com

61–70 of 184 posts

Re: A fork() in the road

#61
post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

[deleted]

Re: A fork() in the road

#62

Fork is quite excellent, except in cases when the intent is to run a different program or when threads are involved (threads are basically an incompatible, competing model of concurrency). The use of fork as a concurrency mechanism (creating a new thread of control that executes in a copy of the address space) is very good and useful. In the POSIX shell language, the subshell syntax (command1; command2; ...) is easil…

The paper agrees with you that the fork models had a reason to exist and that is is perfect for shells.

They also point out that on modern hardware you often should want to write multithreaded multiprocess application.

Their main criticism of fork is that it does not compose at any level of the OS (as it cannot be implemented over a different primitive)

I understand that a lot of people here dislike Microsoft for good reason (not only historical), but drawbacks in fork() are well known and recognized, here they point out that it is also hard-to-impossible to implement as a compatibility layer if the kernel does not support fork.

Also:

> Microsoft "researchers" can stuff it and their company's flagship piece of shit OS.

Do you have any reason to insult Microsoft researchers? They have plenty of citations in this paper of other researchers that appear to agree with them. This type of comments does not appear constructive to me

Re: A fork() in the road

#63
post #32

fork() is also used to daemonize and for privilege separation, two tasks where posix_spawn() cannot be used. I suppose daemonization can be seen as something of the past, but privilege separation is not. On Linux, privileges are attached to a thread, so it should be possible to spawn a new thread instead of a new process. However, a privileged thread sharing the same address space as an unprivileged one doesn't seem…

I can’t answer for Nginx but normally on windows if you want “worker processes” you just start N of them and have them read work from a shared memory queue. That is, workers live longer than the tasks they perform. If one crashes, a new one is spawned. This does seem like a more sensible way of doing things than forking tbh. It isolates work in processes but doesn’t pay for process creation per request.

"If one crashes, a new one is spawned."

I suppose that makes sense on an OS on which crashing is expected behaviour, though some people would want to know what bug caused the crash and whether that bug has security implications.

Re: A fork() in the road

#64

Okay, this one has me laughing out loud. Of COURSE Microsoft doesn't like fork()... Windows pretty much can't do it. I'll admit, there have been a lot of times I wish there was a more streamlined way to spawn processes on Linux (particularly daemons) but when I don't have fork() I always end up missing it. I'd take this paper a lot more seriously if it came from someone with a less obvious bias.

> Windows pretty much can't do it.

Win32 API cannot do it. The underlying NT kernel can.

Re: A fork() in the road

#65

Fork is quite excellent, except in cases when the intent is to run a different program or when threads are involved (threads are basically an incompatible, competing model of concurrency). The use of fork as a concurrency mechanism (creating a new thread of control that executes in a copy of the address space) is very good and useful. In the POSIX shell language, the subshell syntax (command1; command2; ...) is easil…

I realize that "compating" is a misspelling, but I prefer to read it as a portmanteau of "compatible" and "competing" and think it's quite an excellent word for that difficult concept except that it errs slightly too far on the "competing" side.

Yes, "competible" would be good for the opposite situation.

Re: A fork() in the road

#66
post #63

Earlier quoted context omitted.

I can’t answer for Nginx but normally on windows if you want “worker processes” you just start N of them and have them read work from a shared memory queue. That is, workers live longer than the tasks they perform. If one crashes, a new one is spawned. This does seem like a more sensible way of doing things than forking tbh. It isolates work in processes but doesn’t pay for process creation per request.

"If one crashes, a new one is spawned." I suppose that makes sense on an OS on which crashing is expected behaviour, though some people would want to know what bug caused the crash and whether that bug has security implications.

Crashing is an expected behaviour in Linux as well, you can enable coredumps or utilize an applications log if you want to know why.

Re: A fork() in the road

#67
post #19

Earlier quoted context omitted.

This probably isn't the technical explanation your looking for, but, in general, processes on Windows and processes on Unix aren't the same--or, at least, they're not meant to be used the same way. Creating lots of small processes on Windows has long been discouraged and considered poor design, whereas the opposite is true on Unix. One could probably argue that processes on Windows need to be lighter-weight now that…

Windows now has minimal processes that have almost no setup and pico processes (based on minimal processes) that are the foundation for Linux processes in WSL.

The last time I used WSL (perhaps 6 months ago), its per-process overhead was awful. I don't recall the numbers, but I think it managed to start fewer than 10 processes per second. My memory suggests it was more like two processes per second, though I would recommend re-testing before trusting that.

Found my previous comment on it (which has a test case but not numbers): https://news.ycombinator.com/item?id=18226921

Re: A fork() in the road

#68
Can anybody elucidate about why fork() is still used in Chromium or Node.js? They are not old-grown traditional forking Unix servers (unlike Apache or the mentioned databases in the paper). I would expect them to implement some of the alternatives and having fork() only as a fallback in the code (i.e. after a cascade of #ifdefs) if no other API is available. Therefore, I wonder where the fork() bottlenecks really appear in everyday's life.

Re: A fork() in the road

#69
post #44

Earlier quoted context omitted.

And ideally all these things become properties to a configuration object which is then used to spawn a process.

I'm not sure I agree it's the ideal way to do it. That's a heck of a lot of work for one function to do, and it necessarily duplicates the functionality of a ton of other functions. And that's ignoring the fact that forking without ever exec'ing can be really useful in many cases. I haven't yet read the paper, but considering the incredible simplicity from the programmer's PoV that fork provides, and the fact that at…

Less general functions can call more general functions, like fork calls clone on Linux.

Re: A fork() in the road

#70
post #23

Earlier quoted context omitted.

It really doesn't. Microsoft employs a guy named Dave Cutler who is credited with leading the development of Windows NT in the late 80s/early 90s. They hired him from DEC, where he... is credited with co-leading a research project that later became VMS. If you go look at OpenVMS programming manuals, you will see process creation calls (e.g. SYS$CREPRC, LIB$SPAWN) that look and behave a lot like CreateProcess(). I thi…

> I think it's well-known that Windows NT took a lot of ideas from VMS. Has the old theory that "Windows NT" aka "WNT" = "VMS + 1" ever been proved or disproved?

The fact it was called "NT OS/2" before Windows NT seems to indicate it was a happy accident.

(https://americanhistory.si.edu/collections/search/object/nma... the Smithsonian has early design docs with the original name on the spine).

Post reply on HN