In C, how do you know if the dynamic allocation succeeded?

i mean, the world financial system did almost just collapse, not that long ago, https://en.wikipedia.org/wiki/Financial_crisis_of_2007%E2%80... more or less due to the confluence of several adverse events, each of which probably could have been buffered on its own.

When you say additional money would be printed, I assume you mean the M0 money supply would be increased?

I first learned about this while trying to understand processes being killed by OOM in production. We had python 2.x batch jobs being executed by long-running python worker processes -- some of the arbitrary application code in some of the arbitrary batch jobs would occasionally want to execute some command line tool in a new process, and to create the new process under the hood python's subprocess library would fork/exec, and if the parent worker process had already accumulated a large virtual memory footprint, linux's approximate memory accounting heuristics would kick in during the attempted "fork", decide that we were obviously going to run out of physical memory, and kill the process.

We didn't actually want to fork anything and share gigabytes of virtual memory with the child process, we wanted to spawn an almost entirely independent process to do something and report results, but that got implemented under the hood by fork.

Spawning processes is one area where Windows is more elegant than linux: windows offers spawn. Apparently macos and solaris implement a posix_spawn that avoid the complications of fork/exec.

linux offers posix_spawn, apparently which may may or may not call fork under the hood depending on which libc you're using. If libc implements posix_spawn by calling fork then you're back in the same mess with linux heuristic memory accounting and overcommit. E.g. old versions of glibc will fork when you posix_spawn, newer versions of glibc may vfork . musl apparently will always vfork.

It looks like cpython's subprocess.Popen was patched in python 3.8 to detect some cases where posix_spawn can be used -- it reads as if it will only kick in on linux if it detects a sufficiently new version of glibc: https://github.com/python/cpython/blob/main/Lib/subprocess.p...

edit: Python 3.10 now supports using vfork for linux inside subprocess: https://bugs.python.org/issue35823

  docker run --rm -it --entrypoint=/bin/sh python:3.9-alpine
  # apk add strace
  # strace python -c "import subprocess; subprocess.run(['ls', '-l'])" 2>&1 >/dev/null | grep fork
  fork()                                  = 88

  docker run --rm -it --entrypoint=/bin/sh python:3.10-alpine
  # apk add strace
  # strace python -c "import subprocess; subprocess.run(['ls', '-l'])" 2>&1 >/dev/null | grep fork
  vfork()                                 = 15

edit 2: here's a similar tale from go, replacing use of fork in fork/exec:

https://github.com/golang/go/issues/5838

https://go-review.googlesource.com/c/go/+/37439/

https://about.gitlab.com/blog/2018/01/23/how-a-fix-in-go-19-...

I agree with you on all counts, but it's worth highlighting that the parent did not call the designers dumb, but rather the situation and implementation.

I know it can seem like a distinction without a difference, but I think it's fair to critique work. I think they could have been more articulate and considerate towards the designers, and have some empathy that many people have worked very hard on it, and did their best in the problem/solution space they were working with.

I just think it's important that people stay objective on if the quality of the people themselves is in question. It's not ok if it is, but I don't think it was here.

now people that don't check how malloc() returns being blanketly labled as lazy is an example of it being about people. people don't ignore the possibility of a nullptr return from malloc because they're lazy. They ignore it because it's hard -- even if you do catch it, there's very little that you can actually do. You can't dynamically allocate...so I hope you've got enough stack space to do what you've gotta do. And even then, you have to be able to propagate that there's no memory all the way up the stack as it unwinds...and check every single allocation.

The cpp world is mildly better in that it can throw std::bad_alloc...but if you have something that winds up doing an allocation in a destructor, I imagine that's not a fun time.

Most of the time, there's not really a better thing to do than crash -- and there's not a lot of incentive to put any work into it. It's not something that should be happening on any sort of regular basis.

You'd presumably need the parent to also do a system call to mark pages read-write after forking, and that would need to include the stack pages, and that sounds like lots of fun; especially since sometimes library functions fork (although most of that is to fork/exec and more specific apis for that exist now). Not that it isn't loads of fun anyway, but woe betide thee who forks and threads and makes the other threads have a read-only stack.

Probably you'd get people just marking as read-write and returning in sigsegv handlers, which I'm sure has great security properties... OTOH, at least there's an opportunity to deny the remap in the handler and get a decent crashdump from the program or a sliver of hope for managing the situation.

Replying up here instead of way down in the branches.

I don't think disabling of overcommit implies that physical pages are mapped immediately. If caches are instantly droppable, you can use a page that's allocated but unused for cache, and drop the cache page (and zero it) when the allocated page is written to.

You'd still have all of your caches until you have memory pressure with actual data written (but of course, with overcommit, you'd drop caches then too), but if you attempt to allocate more than you have (including through fork attempts as discussed elsewhere), you get a system call failure rather than an OOM kill.