291–300 of 358 posts
It is a weirdly common misconception that that fork() is cheap... it is O(N) on the size of the process, and it always has been. Yes, it's copy on write... but there is a linear relationship between the size of the process and the number of page table entries required to represent it.
This is not exactly fixed since you can vary the amount of memory each page maps with things like hugepages and the same process can run with different page sizes.
Earlier quoted context omitted.
I don't think fork() mandates overcommit. OpenBSD doesn't seem to even allow overcommit or have an OOM killer, memory allocations that exceed available capacity fail immediately even if the memory is not touched.
Let's say you have 1GB RAM. You're running program that occupies 600 MB. Now this program wants to launch second small program that occupies 1 MB. You're doing fork + exec. If you're overcommiting, fork will not reserve another 600 MB, and exec immediately after fork will cause total system usage to be 601 MB. If you're not overcommiting, that fork will fail, because total memory consumption will be 1200 MB which is…
> You're doing fork + exec.
This is the clear problem: you don't want another process that's a duplicate of the current one, that's just a detail of what you actually want: a 1mb process. Right now it's a badly leaky detail which you're forced to work around.
Earlier quoted context omitted.
Don’t pretty much all OSes implement process startup in userspace? On macOS, the kernel creates a process with an image of dyld and points it at dyld_start, which actually takes care of parsing the Mach-O header. I assumed ld.so does the same job on Linux.
Nope, the kernel can load static ELF binaries. ld.so is only needed for dynamically linked binaries, and in fact many Go applications (for example, as they're statically linked) ship as containers with nothing but the single binary.
Earlier quoted context omitted.
A lot of features of UNIX shells are build around pipe and dup and the fork + exec model. One can certainly implement in differently, but it is - like UNIX in general - very nice and elegant.
Help me out here, please. Off the top of my head, the exec command is dependent on exec, except that a spawn + wait implementation would be a mostly okay substitute. Pipes and redirections don’t need fork + exec. Neither do subshells.
Earlier quoted context omitted.
> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returne…
The QNX approach is also pretty much how the dynamic linker loads shared libraries today in Linux . “An era of really expensive memory”. That sounds familiar…
Earlier quoted context omitted.
> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returne…
Cygwin's fork() is similar to what you describe for QNX.
The tricky part is setting up the initial process. The way out for that is static linking and re-use of the fact that the operating system kernel loader has to understand and be able to load (at least a small subset of) program image file formats too.
Earlier quoted context omitted.
Nothing about the UNIX shell is reliant on the fork model. Windows processes have stdio handles as well.
A lot of features of UNIX shells are build around pipe and dup and the fork + exec model. One can certainly implement in differently, but it is - like UNIX in general - very nice and elegant.
Earlier quoted context omitted.
A process that shares nothing with the process that spawned it.
A thing that makes that complicated is that while you want that conceptually, you don't want that in reality. For instance, if the spawning process is in a container of some sort and it spawned a process that "shares nothing with the process that spawned it", the spawned process would no longer be in that container, because the state of "being in the container" is one of the things it shares with the parent process.…
It is a weirdly common misconception that that fork() is cheap... it is O(N) on the size of the process, and it always has been. Yes, it's copy on write... but there is a linear relationship between the size of the process and the number of page table entries required to represent it.
> the number of page table entries This is not exactly fixed since you can vary the amount of memory each page maps with things like hugepages and the same process can run with different page sizes.