Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving beyond fork() + exec()

lwn.net

Post reply on HN
Print view View original

341–350 of 358 posts

Re: Moving beyond fork() + exec()

#341

Post by corbet »

LoganDark wrote: post #257

Huh, LWN has moved to (sometimes) requiring a click to proceed past the subscription pitch to the actual article. I feel like this may have an inverse effect (insistent begging to the point of inserting additional obstacles = angry/insulted users that are less likely to pay).

It's an experiment. Compared to the text-obscuring popovers that are prevalent elsewhere on the net, it seems pretty low-key; as far as I know, this is the first complaint I've seen. I don't know if we will continue experimenting with those or not...better ideas for getting people to subscribe to the site would be more than welcome.

Re: Moving beyond fork() + exec()

#342

Post by cyberax »

pizlonator wrote: post #334
cyberax wrote: post #281

Earlier quoted context omitted.

Unless you want to create a thread in your zygote. Then it breaks down. Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process.

You can create threads in the zygote. It doesn't "break down", but sure, there's a bit more work. My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable, preferably lazily (they resume when they are actually needed). So, the zygotes are sitting with those threads suspended. When they become active, they can do work immediately. They might lazily resume those threads a…

> My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable

Well, yes. You need to wait for all the threads to park themselves at safepoints. This can work if you control the whole runtime, and you don't use something that creates threads behind your back.

This is actually why I've always been interested in a better fork(), it has a lot of parallels with stop-the-world needed for GCs.

> Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives

I don't think we have alternatives? Except maybe ptrace()?

Re: Moving beyond fork() + exec()

#343

Post by simongr3dal »

bluepuma77 wrote: post #197
Animats wrote: post #123

Earlier quoted context omitted.

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returne…

> It was needed back in the era of really expensive memory. Well, it seems we are back in an era with really expensive memory.

That’s funny, but most new cpus today have more L3 cache than those computers had memory and disk space combined.

Re: Moving beyond fork() + exec()

#344

Post by sunshowers »

pjmlp wrote: post #149
sunshowers wrote: post #140

Earlier quoted context omitted.

The problem is that threads are not fault boundaries but processes are. So they're not interchangeable when you care about resilience and misbehaving code.

True, but on Windows the approach is then to use COM servers, which have a faster IPC model, and can even serve multiple clients, depending on how the appartement space is configured.

A one-shot process is easier to build and reason about than an event-driven server (speaking as someone who has written plenty of both).

Re: Moving beyond fork() + exec()

#345

Post by Someone »

thayne wrote: post #215
Someone wrote: post #152

Earlier quoted context omitted.

> The problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1 It may not be slow, but for the common case where fork is almost immediately followed by exec in the process where fork returns zero…

Another possible design is instead of forking the current process, you create a new empty process, then the parent calls syscalls to set up the new process, and eventually call exec on the child process. That does mean you either need new syscalls for that, or adapt existing syscalls to take a pidfd as an argument. That also solves some other problems with fork/exec where the default is to inherit a lot of things you…

I remember reading about an OS where processes weren’t basic building blocks. Instead it had a syscall to create an address space and to create a thread in an address space.

Create a thread in your own address space, and your process becomes multi-threaded. Create an address space, load some code in it, and create a thread there, and you fork/exec-ed.

In my memory, that OS was MACH, but Google doesn’t confirm that for me.

Re: Moving beyond fork() + exec()

#346

Post by uecker »

dwattttt wrote: post #340
uecker wrote: post #317

Earlier quoted context omitted.

Yes, and CreateProcess needs special facilities to make this possible while in the UNIX model you don't.

Which special facilities are you referring to? If it's the ability to selectively inherit handles (fds) to a new process, Linux's lack of this "special facility" is nothing to be proud of. How do you selectively pass on fds without having a global impact on your process?

As explained above, you fork and then before exec you can use all the usual APIs to close/duplicate/... file descriptors. This has no global impact on the parent process because you are already in the child, but one still has access to all the context of the parent needed to whatever configuration you want to do, including things that will be invented in 20 years and are not even conceived today.

Re: Moving beyond fork() + exec()

#347

Post by dwattttt »

uecker wrote: post #346
dwattttt wrote: post #340

Earlier quoted context omitted.

Which special facilities are you referring to? If it's the ability to selectively inherit handles (fds) to a new process, Linux's lack of this "special facility" is nothing to be proud of. How do you selectively pass on fds without having a global impact on your process?

As explained above, you fork and then before exec you can use all the usual APIs to close/duplicate/... file descriptors. This has no global impact on the parent process because you are already in the child, but one still has access to all the context of the parent needed to whatever configuration you want to do, including things that will be invented in 20 years and are not even conceived today.

And as noted elsewhere, what you want is a new process with a specific fd: duplicating your current process state is a path-dependent evolution, rather than a logic step.

There have been plenty of comments here about effective workarounds, multi-process architectures to keep fork cheap, zygotes... these are very specifically working around the problem, while trying to avoid admitting it's a problem.

Re: Moving beyond fork() + exec()

#348

Post by [deleted] »

uecker wrote: post #346
dwattttt wrote: post #340

Earlier quoted context omitted.

Which special facilities are you referring to? If it's the ability to selectively inherit handles (fds) to a new process, Linux's lack of this "special facility" is nothing to be proud of. How do you selectively pass on fds without having a global impact on your process?

As explained above, you fork and then before exec you can use all the usual APIs to close/duplicate/... file descriptors. This has no global impact on the parent process because you are already in the child, but one still has access to all the context of the parent needed to whatever configuration you want to do, including things that will be invented in 20 years and are not even conceived today.

[deleted]

Re: Moving beyond fork() + exec()

#349

Post by uecker »

dwattttt wrote: post #347
uecker wrote: post #346

Earlier quoted context omitted.

As explained above, you fork and then before exec you can use all the usual APIs to close/duplicate/... file descriptors. This has no global impact on the parent process because you are already in the child, but one still has access to all the context of the parent needed to whatever configuration you want to do, including things that will be invented in 20 years and are not even conceived today.

And as noted elsewhere, what you want is a new process with a specific fd: duplicating your current process state is a path-dependent evolution, rather than a logic step. There have been plenty of comments here about effective workarounds, multi-process architectures to keep fork cheap, zygotes... these are very specifically working around the problem, while trying to avoid admitting it's a problem.

You are changing your argument. Let's first agree that my point that being able to use standard APIs (dup,close,pipe,...) is elegant and avoids having a lot of parameter for a process creation call.

Re: Moving beyond fork() + exec()

#350

Post by dwattttt »

uecker wrote: post #349
dwattttt wrote: post #347

Earlier quoted context omitted.

And as noted elsewhere, what you want is a new process with a specific fd: duplicating your current process state is a path-dependent evolution, rather than a logic step. There have been plenty of comments here about effective workarounds, multi-process architectures to keep fork cheap, zygotes... these are very specifically working around the problem, while trying to avoid admitting it's a problem.

You are changing your argument. Let's first agree that my point that being able to use standard APIs (dup,close,pipe,...) is elegant and avoids having a lot of parameter for a process creation call.

I don't think it's an elegant design at all; I think it's a workable design given the constraint of requiring fork.

It's an achievable result with less specialised parts, but there's a reason we don't write all of our logic in NAND gates.

I'm still only inferring what you meant by CreateProcess needing special facilities; I assume you meant opt-in fd inheritance. Saving a parameter call while forcing all fds to be shared/duplicated seems penny-wise but pound-foolish.

Post reply on HN