Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

fork() can fail

rachelbythebay.com

Post reply on HN
Print view View original

281–290 of 320 posts

Re: fork() can fail

#281

Post by swah »

hollerith wrote: post #266
mst wrote: post #262

Earlier quoted context omitted.

The with-current-directory part is the dynamic part - i.e. within its dynamic scope, the current directory is now dir. The scoping of the dir variable itself is irrelevant.

`with-current-directory` has to be a macro because if it is not, then the evaluation of (delete-all-files-recursively) happens before the definition of with-current-directory can change the current directory. What the person meant when he wrote, "In those times I wish I could use the emacs lisp way," is, "In those times I wish I could use a lisp _macro_" -- particularly, one of those macros that makes a change, runs…

Yes, but emacs lisp is the only lisp I used, and lives in an environment where those macros are really useful.

    with-auto-compression-mode  with-case-table
    with-category-table         with-coding-priority
    with-current-buffer         with-decoded-time-value
    with-demoted-errors         with-electric-help
    with-help-window            with-local-quit
    with-no-warnings            with-output-to-string
    with-output-to-temp-buffer  with-selected-frame
    with-selected-window        with-silent-modifications
    with-syntax-table           with-temp-buffer
    with-temp-buffer-window     with-temp-file
    with-temp-message           with-timeout
    with-timeout-suspend        with-timeout-unsuspend
    with-wrapper-hook

Re: fork() can fail

#282

Post by mercurial »

Alupis wrote: post #276
ajkjk wrote: post #176

Earlier quoted context omitted.

There'd be far fewer bugs if everyone knew exactly how everything else works. This kind of mistake is godawful and should not be defended. (but it's correctly fixed through stronger typing, not through choosing -48585 as the code for killing everything).

> not through choosing -48585 as the code for killing everything) Actually, only -1 is the code that "kills everything" > There'd be far fewer bugs if everyone knew exactly how everything else works. Perhaps I misinterpreted your meaning, because you seem to be advocating using programming and scripting languages without actually bothering to learn them. Of course this can, will and does lead to very bad effects. The…

> Perhaps I misinterpreted your meaning, because you seem to be advocating using programming and scripting languages without actually bothering to learn them. Of course this can, will and does lead to very bad effects.

You're arguing that people should read the API before doing anything with it. Parent's point is that this class of error can be avoided by strong typing (eg, via algebraic datatypes), negating the chance that it would happen in the first place. Which, I think, is the right way to look at the problem. But certainly, if you do have to use a weakly-typed, unsafe language which does not provide this kind of guarantee, be sure to read the documentation twice.

Which doesn't mean you won't get bitten when it turns out that the person writing a library you rely didn't RTFM.

Re: fork() can fail

#283

Post by derefr »

zurn wrote: post #225
derefr wrote: post #213

Earlier quoted context omitted.

> ...a minute or two later one of the folks who had 'root' ran into the machine room with a panic-stricken look because the system had mostly just locked up. It's kind of weird that, while root has always had e.g. 5% reserved disk space on the rootfs for emergencies, one thing no Unix has ever done is enforce a 5% CPU reservation for root so administrators can "talk over" a cascading failure. I think this is possible…

It's not specifically the lack of cpu timeslices that crowds out other programs, it's more like exhaustion of all the OS resources (process table fills up, file table fills up, memory runs out, swap death etc). Sure if you carefully made everything fork-bomb-resistant then a cpu quota would be a part of it. Container systems use fork bombs as basic test cases.

I'm surprised that this wasn't one of the primary goals of cgroups: the ability to group "all userspace processes" into one cgroup, and then say that that cgroup can in sum only use so much CPU, so many processes, so many inodes, etc. You know, a control plane/data plane separation, without requiring hypervision.

Re: fork() can fail

#284

Post by derefr »

paulfurtado wrote: post #236
derefr wrote: post #213

Earlier quoted context omitted.

> ...a minute or two later one of the folks who had 'root' ran into the machine room with a panic-stricken look because the system had mostly just locked up. It's kind of weird that, while root has always had e.g. 5% reserved disk space on the rootfs for emergencies, one thing no Unix has ever done is enforce a 5% CPU reservation for root so administrators can "talk over" a cascading failure. I think this is possible…

Most linux distributions assign root processes a better scheduling priority than non-root processes, which should be good enough in most cases. Critical system processes also run at better priorities than other processes. It's not uncommon to see linux users consciously decide on the priority of a process by using nice or renice. Totally limiting the CPU utilization of a group of processes requires more overhead than…

> In any case, there will likely always be some system call you can abuse to totally exhaust some resource of the kernel.

If this is true, I would expect there to exist one or more articles entitled "how I brought down my Heroku host-instance" or something along those lines. Anyone got some links? :)

Re: fork() can fail

#285

Post by roryokane »

Alupis wrote: post #275
jwr wrote: post #247

Earlier quoted context omitted.

You do realize that this, in turn, creates a race condition, right?

? not at all -- you make the deletion of the directory dependent on whether or not you tested true for the directory being present. I don't see why this would cause any race condition...

The race condition is when another program is interacting with that directory too. So you check that the directory exists, and then the other program gets CPU time and uses it to delete that directory, and then your program tries to delete that directory but fails.

To avoid this, you have to either get a lock on that directory somehow (opening a transaction), or you have to try deleting the directory and then check the return code or catch any exceptions to tell whether the directory existed at the time of the attempted deletion.

Re: fork() can fail

#286

Post by AnimalMuppet »

djcapelis wrote: post #163
__david__ wrote: post #143

Earlier quoted context omitted.

> Note however, that strictly speaking stderr does not have to be 2. It Nope. On POSIX systems stderr is defined as 2: http://pubs.opengroup.org/onlinepubs/9699919799/functions/st... > The following symbolic values in define the file descriptors > that shall be associated with the C-language stdin, stdout, and stderr > when the application is started: > > STDIN_FILENO > Standard input value, stdin. Its value is 0. >…

Not quite. That is according to POSIX, but not the C standard. stderr is defined by the C standard. POSIX is a standard followed by many of the systems that run C. Strictly speaking, stderr does not have to be 2. It wouldn't comply with POSIX in that case, but it would still be C.

Note, however, that these were Unix systems where we encountered the problem, so they should have been POSIX-compliant.

Re: fork() can fail

#287

Post by kjs3 »

kabdib wrote: post #209

When I was young and really didn't understand Unix, my friend and were summer students at NBS (now NIST), and one fine afternoon we wondered what would happen if you ran fork() forever. We didn't know, so we wrote the program and ran it. This was on a PDP-11/45 running v6 or v7 Unix. The printing console (some DECWriter 133 something or other) started burping and spewing stuff about fork failing and other bad things,…

Must have been V6; I recall V7 had patches to prevent this, at least to the extent that it wouldn't crater the whole machine. I haven't thought about using ncheck & icheck since fsdb showed up about BSD4.2 or thereabouts. I remember using adb as well to fix buggered filesystems back in the ancient days.

I remember well the day one of the elder neckbeards handed me my own photocopy of the Lions books. It was enlightenment in pure form.

Re: fork() can fail

#288

Post by peterwwillis »

dredmorbius wrote: post #224
mjevans wrote: post #216

Earlier quoted context omitted.

It would only be possible if a limit were enforced on all non-primary namespaces. However something that has been /possible/ for a while (but not in practice done) would be to elevate root process priority over other processes. Probably not done due to daemons needing to run as root (which is decreasing as they're able to drop privileges these days).

Root has had the ability to assign negative nice values since long, long ago. Non-root users can only assign positive niceness. The range is -20 - +19. In theory this can give higher priority to a process, but if you cannot get into the run-queue at all (fork bomb), or the problem is in kernel space (e.g., I/O access, hang, or a kernel space loop), then it's not going to help you much.

And, sadly, most of the really hard hangs are kernel space. The general fix is to cut off all network requests/incoming jobs, powercycle, dig through logs, and try to shunt a future hang. (Sometimes just cutting incoming jobs will stop the hang, too)

Re: fork() can fail

#289

Post by jrockway »

stinos wrote: post #14
cperciva wrote: post #7

This reminds me of one of the most epic bugs I've ever run into: mkdir("/foo", 0700); chdir("/foo"); recursively_delete_everything_in_current_directory(); Running as root, this usually worked fine: It would create a directory, move into it, and clean out any garbage left behind by a previous run before doing anything new. Running as non-root, the mkdir failed, the chdir failed, and it started eating my home directory…

When you see chdir, or any notion of the current working directory being used for anything: run as fast as you can. (or refactor if it's not too late). Things I've seen because of software relying on it.. Sometimes it's just directories/files it creates popping up all over the place, sometimes it's 'just' crashing, but yes sometimes it starts to erase and all hell really breaks loose.

I chdir("/") before daemonizing so that the filesystem the process was using as its cwd can be unmounted :)

Re: fork() can fail

#290

Post by peterwwillis »

stinos wrote: post #14
cperciva wrote: post #7

This reminds me of one of the most epic bugs I've ever run into: mkdir("/foo", 0700); chdir("/foo"); recursively_delete_everything_in_current_directory(); Running as root, this usually worked fine: It would create a directory, move into it, and clean out any garbage left behind by a previous run before doing anything new. Running as non-root, the mkdir failed, the chdir failed, and it started eating my home directory…

When you see chdir, or any notion of the current working directory being used for anything: run as fast as you can. (or refactor if it's not too late). Things I've seen because of software relying on it.. Sometimes it's just directories/files it creates popping up all over the place, sometimes it's 'just' crashing, but yes sometimes it starts to erase and all hell really breaks loose.

The exception is chdir("/"). This will always give you the desired behavior, only error on EACCES, and prevent many nasty filesystem problems. This also removes the idea of relative paths, so the user is forced to use full paths (or, every relative path is a full path). Not very practical but it does work.
Post reply on HN