Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A fork() in the road

microsoft.com

Post reply on HN
Print view View original

161–170 of 184 posts

Re: A fork() in the road

#161

Post by nitwit005 »

evilotto wrote: post #31
swiftcoder wrote: post #8

Fork has really weird semantics, and a lot of fun gotchas around managing resources. Good riddance?

Not even just the semantics, the performance is awful. Even when the fork is virtual (as any modern fork is) and there's no memory copying because it's COW, all the kernel page tables still need to be copied and for a multi-GB process that's nontrivial. That's why any sane large service that needs to fork anything will early on start up a slave subprocess whose only job is to fork quickly when the master process need…

These days, you can usually start a process without forking through posix_spawn/vfork. Although, I gather some servers still do it so they can set the current working directory more easily.

Re: A fork() in the road

#162

Post by Dylan16807 »

cryptonector wrote: post #132
eesmith wrote: post #46

Earlier quoted context omitted.

When you read the paper, you'll see this covered in section 6 ("REPLACING FORK") subsection "Low-level: Cross-process operations" > While a spawn-like API is preferred for most instances of starting a program, for full generality it requires a flag, parameter, or new helper function controlling every possible aspect of process state. It is infeasible for a single OS API to give complete control over the initial state…

That has security considerations when spawning a process to run an executable that gets privilege on exec (think set-uid on Unix).

Easy to deal with by not applying privileges until the parent is done tinkering and hits start.

Re: A fork() in the road

#163

Post by kazinator »

afiori wrote: post #110
kazinator wrote: post #86

Earlier quoted context omitted.

Virtual machines can be forked processes, and contain operating systems with forked processes, some of which are virtual machiens ... fork composes!

as a function obviously, the point is that it does not compose easily with other abstractions. That is every other library and OS functionality needs to be fork-aware. spawn do not have this requirement.

The concept of "fork aware" didn't exist until threads. You could argue it's a thread problem. Remember, every library and OS functionality aso needs to be "thread aware" when threads are introduced. The pthread_atfork function can be thought about as "what do we do about thread and thread paraphernalia when we fork" rather than "what do we do about fork when we have threads".

Even the close-on-exec flag race condition is a result of threads. duplicating a file descriptor and setting its close-on-exec flag is a two step process during which a fork can happen, causing a child to inherit the descriptor without close-on-exec flag being yet set. But that can only happen if there are threads. (Or something crazy, like fork being called out of an async signal handler).

Re: A fork() in the road

#164

Post by kazinator »

afiori wrote: post #62
kazinator wrote: post #54

Fork is quite excellent, except in cases when the intent is to run a different program or when threads are involved (threads are basically an incompatible, competing model of concurrency). The use of fork as a concurrency mechanism (creating a new thread of control that executes in a copy of the address space) is very good and useful. In the POSIX shell language, the subshell syntax (command1; command2; ...) is easil…

The paper agrees with you that the fork models had a reason to exist and that is is perfect for shells. They also point out that on modern hardware you often should want to write multithreaded multiprocess application. Their main criticism of fork is that it does not compose at any level of the OS (as it cannot be implemented over a different primitive) I understand that a lot of people here dislike Microsoft for goo…

> Do you have any reason to insult Microsoft researchers?

Yes: they are pompous wankers who want to dictate not only what features people should include in their operating system, but what we should be teaching CS students.

Re: A fork() in the road

#165

Post by kazinator »

JdeBP wrote: post #81
kazinator wrote: post #75

Earlier quoted context omitted.

CreateProcess requires an application to initialize from scratch. When you fork, you cheaply inherit the initialized state of the whole application image. Only a few pages that are mutated have to be subject to copy-on-write. Even that copy-on-write is cheaper than calculating the contents of those pages from scratch.

There has been a lot of discussion in recent years about how cheap that "cheaply" really is. * https://news.ycombinator.com/item?id=9653238 * https://news.ycombinator.com/item?id=18071278 * https://news.ycombinator.com/item?id=19622503

fork is pretty much always going to be cheaper than starting a new process in scratch over the same executable image (and library images) and then re-playing everything inside that process so that it gets into exactly the same state as the creator to be a de facto clone of it.

Re: A fork() in the road

#166

Post by Dylan16807 »

bloak wrote: post #157
zaarn wrote: post #66

Earlier quoted context omitted.

Crashing is an expected behaviour in Linux as well, you can enable coredumps or utilize an applications log if you want to know why.

The Linux kernel doesn't crash much, unless you have dodgy drivers or dodgy hardware. Whether your userland programs crash or not depends on what you're running. I don't expect to see sshd crashing, for example, though it's true that almost any program will exit suddenly if the system runs out of memory, which to an ordinary user looks like a crash, though it's a very different thing really.

If you weren't talking about userland crashes, then your crack about "an OS on which crashing is expected behavior" makes no sense.

Re: A fork() in the road

#167

Post by a1369209993 »

Dylan16807 wrote: post #162
cryptonector wrote: post #132

Earlier quoted context omitted.

That has security considerations when spawning a process to run an executable that gets privilege on exec (think set-uid on Unix).

Easy to deal with by not applying privileges until the parent is done tinkering and hits start.

Specifically, you'd want to do something like:

  pid_t child = pfork();
  for(int fd=0;fd
(Y'know, this looks kind of familiar...)

Re: A fork() in the road

#168

Post by a1369209993 »

IshKebab wrote: post #154
zzzcpan wrote: post #91

It's hard to take them seriously when they imply the mess that threads are is somehow acceptable and necessary, but nicer, less error prone and simpler fork isn't. Threads are a nasty hack and a liability for the modern programmer to use. And systems researchers really should acknowledge that their continued existence as first class OS primitives is holding back systems research much more, than fork. I guess they are…

What's wrong with threads exactly?

Aliasable, mutable memory (ie race conditions) is evil, and threads perfuse the entire programming environment with it. This is a dirty implementation detail that operating system kernels have to deal with, and we should be burying it in the same hole as memory swapping and TCP retransmits, not making it a fundamental hazard every application developer has to worry about.

Re: A fork() in the road

#169

Post by Dylan16807 »

xyzzyz wrote: post #88
ktpsns wrote: post #68

Can anybody elucidate about why fork() is still used in Chromium or Node.js? They are not old-grown traditional forking Unix servers (unlike Apache or the mentioned databases in the paper). I would expect them to implement some of the alternatives and having fork() only as a fallback in the code (i.e. after a cascade of #ifdefs) if no other API is available. Therefore, I wonder where the fork() bottlenecks really app…

> why fork() is still used in Chromium To support a multi-process web browser architecture that Chromium pioneered, you need to spawn processes. See https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...

That's not what the page says. It says the use of fork() saves 8MB and a few tens of milliseconds per process spawn.

Re: A fork() in the road

#170

Post by Dylan16807 »

a1369209993 wrote: post #167
Dylan16807 wrote: post #162

Earlier quoted context omitted.

Easy to deal with by not applying privileges until the parent is done tinkering and hits start.

Specifically, you'd want to do something like: pid_t child = pfork(); for(int fd=0;fd (Y'know, this looks kind of familiar...)

I'm really not sure what you're implying, can you please state it explicitly? I'd expect this code to look similar to both the CreateProcess and fork models, at the very least.
Post reply on HN