In C, how do you know if the dynamic allocation succeeded?

Post by jagrsw » Sat, Nov 06, 2021, 8:12 AM UTC

bitwize wrote: post #99

joosters wrote: post #41

Earlier quoted context omitted.

Please don’t blindly declare it ‘totally dumb’. If you disallow overcommit, you can end up with a system that can’t fork and run /bin/true, even if there are gigabytes of memory left. Both styles of memory allocation have their uses, and their drawbacks, but please understand them before declaring many OS designers as stupid and dumb.

That's just the thing. fork(2), like much of POSIX, was a mistake. The proper semantics for starting a new process is something like posix_spawn or Win32 CreateProcess, i.e., you specify an executable image to start.

  alarm(0); /* disable alarms */
  setenv(); /* A couple of required envs, like MALLOC_PERTURB_ or MALLOC_PERTURB_ */
  prctl(PR_SET_DUMPABLE, 1) /* regarding ptrace()-attach */
  syscall(__NR_personality, ADDR_NO_RANDOMIZE); /* disable ASLR for debugging, if needed */
  socketpair() /* reliable execve success detection, some form of witchcraft */
  setpriority()
  prctl(PR_SET_PDEATHSIG, SIGKILL); /* die upon parents death */
  setrlimit(); /* set of reset rlimits */
  lseek(fd, 0, SEEK_SET); /* rewind input file for this specific subprocess */
  /* prepare arguments (argv) for execve dynamically */
  sysconf(_SC_NPROCESSORS_ONLN); pthread_setaffinity_np(); /* pin subprocess to a list of CPUs */
  /*
  LOTS of functions here
  if we wanted to use net/process/mount namespacing
  e.g:
  assigning IP adddresses to interfaces
  creating custom views of the filesystem tree
  modifying capability sets
  */
  open("/proc/self/oom_score_adj"), write(), close(); /* adjustment of oom score */
  open("/proc/self/fd", O_DIRECTORY); getdents(); fcntl(F_GETFD); fcntl(F_SETFD, FD_CLOEXEC); close() /* closing fds upon exec */
  setsid(); /* new session */
  sigprocmask(empty_set); /* reset signal mask */
  open("/dev/null"); dup2(null, 0..1); /* close fd 0,1,2 */
  prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0)
  prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER); /* application of sandboxing */
  and finally execv() or execveat()

Post by rurban » Sat, Nov 06, 2021, 8:14 AM UTC

kibwen wrote: post #21

mirashii wrote: post #2

I'm a little disappointed that the article didn't answer the question, or at least try to. A discussion of using read/write vs mincore vs trying to catch a SIGSEGV would've been a nice addition.

The answer is that you kind of can't. You're at the mercy of the OS to give you accurate information, and malloc as an interface isn't set up to distinguish between virtual memory and "actual" memory. We could imagine a separate interface that would allow the OS to communicate this distinction (or hacks like you allude to), but I don't know of any standard approach.

Post by carlmr » Sat, Nov 06, 2021, 8:28 AM UTC

DeathArrow wrote: post #149

carlmr wrote: post #144

Earlier quoted context omitted.

K8s might save you from crashes affecting availability but what about data corruption, logicals errors sound APIs etc.? I'm guessing if code quality is neglected all of these suffer.

Code quality is not neglected to the point we have logical errors. Architecture is reasonable and we have an extensive suite of end to end tests and integration tests.

Post by lokedhs » Sat, Nov 06, 2021, 8:42 AM UTC

pcwalton wrote: post #69

convolvatron wrote: post #17

Earlier quoted context omitted.

wouldn't you just account the COW pages against the parent until they are copied? kicking the can down the road means there isn't any longer a reasonable correction (failing the allocation), but instead we get to drive around randomly trying to find something to kill. this is particularly annoying if you are running a service. there is no hope for it to recover - for example by flushing a cache. instead the OS looks…

> wouldn't you just account the COW pages against the parent until they are copied? That's what Linux does. But how do you return ENOMEM when the copy does happen and now the system is out of memory? Memory writes don't return error codes. The best you could do is send a signal, which is exactly what the OOM killer does.

Post by Koromix » Sat, Nov 06, 2021, 8:51 AM UTC

jagrsw wrote: post #151

bitwize wrote: post #99

Earlier quoted context omitted.

That's just the thing. fork(2), like much of POSIX, was a mistake. The proper semantics for starting a new process is something like posix_spawn or Win32 CreateProcess, i.e., you specify an executable image to start.

> The proper semantics for starting a new process is.. :) Ah.. I guess when "I was younger" (TM) I was also so dogmatic on most things computer-related. Anyways, when we need to do something a bit more complex than equivalent of system(), then it quickly becomes evident, that in many cases we need to prepare ourselves for the future execve(). Here's the list of syscalls/c-funcs, which are called in two random project…

  int process_handle = spawn("/bin/true", SP_PAUSED); // create process but don't execute, and return a handle
  // most API would take a process handle, thus you could do stuff (such as prctl) on the new process
  unpause(process_handle);