Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A fork() in the road

microsoft.com

Post reply on HN
Print view View original

81–90 of 184 posts

Re: A fork() in the road

#81

Post by JdeBP »

kazinator wrote: post #75
sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

CreateProcess requires an application to initialize from scratch. When you fork, you cheaply inherit the initialized state of the whole application image. Only a few pages that are mutated have to be subject to copy-on-write. Even that copy-on-write is cheaper than calculating the contents of those pages from scratch.

There has been a lot of discussion in recent years about how cheap that "cheaply" really is.

* https://news.ycombinator.com/item?id=9653238

* https://news.ycombinator.com/item?id=18071278

* https://news.ycombinator.com/item?id=19622503

Re: A fork() in the road

#82

Post by pjc50 »

chungleong wrote: post #76
richardwhiuk wrote: post #45

Earlier quoted context omitted.

Pretty much identical to shared objects on Linux

Windows DLLs require fixups when they're loaded off their preferred base address.

So do relocatable shared libraries on Linux. https://eli.thegreenplace.net/2011/08/25/load-time-relocatio...

Re: A fork() in the road

#83

Post by JdeBP »

the_mitsuhiko wrote: post #39
spc476 wrote: post #37

Earlier quoted context omitted.

Between the `fork()` and an `exec()`, I can: * redirect stdin, stdout, and stderr * open files that might be needed and close files that aren't * change process limits * drop privileges * change the root directory * change namespaces And there are a few other things I am probably forgetting.

And ideally all these things become properties to a configuration object which is then used to spawn a process.

You could call it posix_spawn_file_actions_t and posix_spawnattr_t . (-:

Re: A fork() in the road

#84

Post by mikekchar »

alkonaut wrote: post #57
vbernat wrote: post #32

fork() is also used to daemonize and for privilege separation, two tasks where posix_spawn() cannot be used. I suppose daemonization can be seen as something of the past, but privilege separation is not. On Linux, privileges are attached to a thread, so it should be possible to spawn a new thread instead of a new process. However, a privileged thread sharing the same address space as an unprivileged one doesn't seem…

I can’t answer for Nginx but normally on windows if you want “worker processes” you just start N of them and have them read work from a shared memory queue. That is, workers live longer than the tasks they perform. If one crashes, a new one is spawned. This does seem like a more sensible way of doing things than forking tbh. It isolates work in processes but doesn’t pay for process creation per request.

I may be strange, but that's the way I've always used fork() as well. It's one of the reasons why named pipes exist (or at least that's what I've always thought).

Re: A fork() in the road

#85

Post by afiori »

kazinator wrote: post #74
afiori wrote: post #62

Earlier quoted context omitted.

The paper agrees with you that the fork models had a reason to exist and that is is perfect for shells. They also point out that on modern hardware you often should want to write multithreaded multiprocess application. Their main criticism of fork is that it does not compose at any level of the OS (as it cannot be implemented over a different primitive) I understand that a lot of people here dislike Microsoft for goo…

It's an idiotic argument. Only functions compose. Though fork is packaged as a function, it's really an operator with a big effect. Booting a system doesn't compose; let's not have power-on reset and bootloaders. Everything in this paper could have been cribbed from twenty year or older Usenet postings, mailing lists and other sources. Fork has been dissected ad nausem ; anyone who is anyone in the Unix-like world kn…

Also levels of abstraction compose.

> Booting a system doesn't compose;

Actually this is false, virtual machine and hypervisors allow to boot a system inside another system

Re: A fork() in the road

#86

Post by kazinator »

afiori wrote: post #85
kazinator wrote: post #74

Earlier quoted context omitted.

It's an idiotic argument. Only functions compose. Though fork is packaged as a function, it's really an operator with a big effect. Booting a system doesn't compose; let's not have power-on reset and bootloaders. Everything in this paper could have been cribbed from twenty year or older Usenet postings, mailing lists and other sources. Fork has been dissected ad nausem ; anyone who is anyone in the Unix-like world kn…

Also levels of abstraction compose. > Booting a system doesn't compose; Actually this is false, virtual machine and hypervisors allow to boot a system inside another system

Virtual machines can be forked processes, and contain operating systems with forked processes, some of which are virtual machiens ... fork composes!

Re: A fork() in the road

#87

Post by sqrt17 »

dis-sys wrote: post #26

I agree with most parts of the paper. Fork() is now basically the root of a looong list of special cases in so many aspects of programming. Things get even worse when you use a language with built-in runtime such as Golang for which multi-threaded programming the default behaviour. If fork() can't even handle multiple threads, what is the real point of having it when a 8 core 16 threads AMD processor is about $150 ea…

> If fork() can't even handle multiple threads, what is the real point of having it when a 8 core 16 threads AMD processor ...

These threads and those threads are not the same. The 16-threads SMT processor will happily chew on 16 different programs, processes or whatever the load at the moment is, e.g. if you use Python's multiprocessing you can create 16 processes and they'll be executed in parallel.

fork() can handle multiple threads but you have to be attentive when cleaning up etc. - quite often, code using fork() will get confused when you spawn threads, and code using threads will get confused when you fork()

Re: A fork() in the road

#88

Post by xyzzyz »

ktpsns wrote: post #68

Can anybody elucidate about why fork() is still used in Chromium or Node.js? They are not old-grown traditional forking Unix servers (unlike Apache or the mentioned databases in the paper). I would expect them to implement some of the alternatives and having fork() only as a fallback in the code (i.e. after a cascade of #ifdefs) if no other API is available. Therefore, I wonder where the fork() bottlenecks really app…

> why fork() is still used in Chromium

To support a multi-process web browser architecture that Chromium pioneered, you need to spawn processes. See https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...

Re: A fork() in the road

#90

Post by intea »

ralish wrote: post #21
sfink wrote: post #17

I read the paper, and they make a lot of good points about fork's warts. But I really wanted some explanation of why Windows process startup seems to be so heavyweight. Why does anything that spawns lots of little independent processes take so bloody long on Windows? I'm not saying "lots of processes on Windows is slow, lots of processes on Linux is fast, Windows uses CreateProcess, Linux uses fork, CreateProcess is…

I'm a bit rusty on this but from memory the overhead is by and large specific to the Win32 environment. Creating a "raw" process is cheap and fast (as you'd reasonably expect), but there's a lot of additional initialisation that needs to occur for a "fully-fledged" Win32 process before it can start executing. Beyond the raw Process and Thread kernel objects, which are represented by EPROCESS + KPROCESS and ETHREAD +…

The WSL processes are called pico processes.

https://blogs.msdn.microsoft.com/wsl/2016/05/23/pico-process...

Post reply on HN