Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A fork() in the road

microsoft.com

Post reply on HN
Print view View original

21–30 of 184 posts

Re: A fork() in the road

#21

Post by ralish »

sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

I'm a bit rusty on this but from memory the overhead is by and large specific to the Win32 environment. Creating a "raw" process is cheap and fast (as you'd reasonably expect), but there's a lot of additional initialisation that needs to occur for a "fully-fledged" Win32 process before it can start executing.

Beyond the raw Process and Thread kernel objects, which are represented by EPROCESS + KPROCESS and ETHREAD + KTHREAD structures in kernel address space, a Win32 process also needs to have:

- A PEB (Process Environment Block) structure in its user address space

- An associated CSR_PROCESS structure maintained by Csrss (Win32 subsystem user-mode)

- An associated W32PROCESS structure for Win32k (Win32 subsystem kernel-mode)

I'm pretty sure these days the W32PROCESS structure only gets created on-demand with the first creation of a GDI or USER object, so presumably CLI apps don't have to pay that price. But either way, those latter three structures are non-trivial. They are complicated structures and I assume involve a context switch (or several) at least for the Csrss component. At least some steps in the process also involve manipulating global data structures which block other process creation/destruction (Csrss steps only?).

I expect all this Win32 specific stuff largely doesn't apply to e.g. the Linux subsystem, and so creating processes should be much faster. The key takeaway is its all the Win32 stuff that contributes the bulk of the overhead, not the fundamental process or thread primitives themselves.

EDIT: If you want to learn more, Mark Russinovich's Windows Internals has a whole chapter on process creation which I'm sure explains all this.

Re: A fork() in the road

#22

Post by eesmith »

swiley wrote: post #13
Solomoriah wrote: post #2

Okay, this one has me laughing out loud. Of COURSE Microsoft doesn't like fork()... Windows pretty much can't do it. I'll admit, there have been a lot of times I wish there was a more streamlined way to spawn processes on Linux (particularly daemons) but when I don't have fork() I always end up missing it. I'd take this paper a lot more seriously if it came from someone with a less obvious bias.

I thought Linux had clone which glibc called for their implementation of fork.

Section 6: REPLACING FORK

> Alternative: clone().

> This syscall underlies all process and thread creation on Linux. Like Plan 9’s rfork() which preceded it, it takes separate flags controlling the child’s kernel state: address space, file descriptor table, namespaces, etc. This avoids one problem of fork: that its behaviour is implicit or undefined for many abstractions. However, for each resource there are two options: either share the resource between parent and child, or else copy it. As a result, clone suffers most of the same problems as fork (§4–5).

Re: A fork() in the road

#23

Post by saxonww »

lawl wrote: post #9
saagarjha wrote: post #6

Earlier quoted context omitted.

It's suggesting posix_spawn, which is standardized and has nothing to do with Microsoft.

> Just as a programming course would not today begin with goto, we suggest teaching either posix_spawn() or CreateProcess(), and then introducing fork as a special case with its historic context (§2). Or CreateProcess(), which has a lot to do with microsoft.

It really doesn't. Microsoft employs a guy named Dave Cutler who is credited with leading the development of Windows NT in the late 80s/early 90s. They hired him from DEC, where he... is credited with co-leading a research project that later became VMS. If you go look at OpenVMS programming manuals, you will see process creation calls (e.g. SYS$CREPRC, LIB$SPAWN) that look and behave a lot like CreateProcess().

I think it's well-known that Windows NT took a lot of ideas from VMS.

https://en.wikipedia.org/wiki/Dave_Cutler https://docs.microsoft.com/en-us/windows/desktop/api/process... https://www.itec.suny.edu/scsys/vms/OVMSDOC073/v73/5932/5932... http://www.itec.suny.edu/scsys/vms/OVMSDOC073/v73/5841/5841p...

I don't think we should ever forget how MS behaved through the mid 2000s. But we don't live in that world anymore, they aren't (capable of being) that company anymore, and I think we're at a point where dismissing research because of a connection to MS is not protecting anyone from anything.

Re: A fork() in the road

#24

Post by zenexer »

swiley wrote: post #13
Solomoriah wrote: post #2

Okay, this one has me laughing out loud. Of COURSE Microsoft doesn't like fork()... Windows pretty much can't do it. I'll admit, there have been a lot of times I wish there was a more streamlined way to spawn processes on Linux (particularly daemons) but when I don't have fork() I always end up missing it. I'd take this paper a lot more seriously if it came from someone with a less obvious bias.

I thought Linux had clone which glibc called for their implementation of fork.

Yes, the underlying syscall for fork() is clone [1], and the underlying syscall for exec*() is execve [2].

[1]: http://man7.org/linux/man-pages/man2/fork.2.html#NOTES

[2]: http://man7.org/linux/man-pages/man2/execve.2.html

Re: A fork() in the road

#25

Post by wvenable »

zenexer wrote: post #19
sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

This probably isn't the technical explanation your looking for, but, in general, processes on Windows and processes on Unix aren't the same--or, at least, they're not meant to be used the same way. Creating lots of small processes on Windows has long been discouraged and considered poor design, whereas the opposite is true on Unix. One could probably argue that processes on Windows need to be lighter-weight now that…

Windows now has minimal processes that have almost no setup and pico processes (based on minimal processes) that are the foundation for Linux processes in WSL.

Re: A fork() in the road

#26

Post by dis-sys »

I agree with most parts of the paper.

Fork() is now basically the root of a looong list of special cases in so many aspects of programming. Things get even worse when you use a language with built-in runtime such as Golang for which multi-threaded programming the default behaviour. If fork() can't even handle multiple threads, what is the real point of having it when a 8 core 16 threads AMD processor is about $150 each.

Re: A fork() in the road

#27

Post by Dylan16807 »

tomjakubowski wrote: post #15
saagarjha wrote: post #10

Earlier quoted context omitted.

Yeah, if you're using Windows you aren't going to be able to use it. Or are you suggesting that Microsoft should implement posix_spawn?

Can't you use posix_spawn() with WSL and your favorite POSIX-compatible libc implementation?

Well that's a complicated question to answer.

You can use the posix_spawn function in glibc, which uses a vfork or clone syscall just like on Linux.

Re: A fork() in the road

#28

Post by evilotto »

saxonww wrote: post #23
lawl wrote: post #9

Earlier quoted context omitted.

> Just as a programming course would not today begin with goto, we suggest teaching either posix_spawn() or CreateProcess(), and then introducing fork as a special case with its historic context (§2). Or CreateProcess(), which has a lot to do with microsoft.

It really doesn't. Microsoft employs a guy named Dave Cutler who is credited with leading the development of Windows NT in the late 80s/early 90s. They hired him from DEC, where he... is credited with co-leading a research project that later became VMS. If you go look at OpenVMS programming manuals, you will see process creation calls (e.g. SYS$CREPRC, LIB$SPAWN) that look and behave a lot like CreateProcess(). I thi…

> I think it's well-known that Windows NT took a lot of ideas from VMS.

Has the old theory that "Windows NT" aka "WNT" = "VMS + 1" ever been proved or disproved?

Re: A fork() in the road

#29

Post by DannyBee »

Solomoriah wrote: post #2

Okay, this one has me laughing out loud. Of COURSE Microsoft doesn't like fork()... Windows pretty much can't do it. I'll admit, there have been a lot of times I wish there was a more streamlined way to spawn processes on Linux (particularly daemons) but when I don't have fork() I always end up missing it. I'd take this paper a lot more seriously if it came from someone with a less obvious bias.

As the article points out, the NT kernel actually natively supports fork. It's just not exposed.

Re: A fork() in the road

#30

Post by muststopmyths »

sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

If I had to guess, I'd point to DLLs. The minimal Windows process loads probably half a dozen, plus the entry points are called in a serialized manner.
Post reply on HN