Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

71–80 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#71

Post by nomel »

kbolino wrote: post #62
shikon7 wrote: post #24

I wonder why it is so hard for Rust to implement its own safe stdlib independent of C.

They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.

> and environment variables require an operating system

Is that true? It's just a process global string -> string map, that can be pre-loaded with values before the process starts, with a copy of the current state being passed to any sub-process. This could be trivially implemented with batch processing/supervisory programs.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#72

Post by docandrew »

plorkyeran wrote: post #55
msully4321 wrote: post #11

Earlier quoted context omitted.

Yeah, setenv should probably just not exist, and environment variables should be only set when spawning new processes.

The problem is that applications sometimes need to set environment variables which will be read by libraries in the same process. This is safe to do during startup, but at no later times. Ideally all libraries which use environment variables should have APIs allowing you to override the env variables without calling setenv(), but that isn't always the case.

I’d argue that libraries shouldn’t read environment variables at all. They’re passed on the initial program stack and look just like stack vars, so the issue here is essentially the same as taking the address of a stack variable and misusing it.

Just like a library wouldn’t try to use argv directly, it shouldn’t use envp either (even if done via getenv/setenv)

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#73

Post by viraptor »

sim7c00 wrote: post #23
forrestthewoods wrote: post #3

Mutable global state is evil. Friends don’t let friends use mutable global state. I hate envvars. It’s “the Linux way”. I avoid them like the plague. A++ strong recommend. libc is terrible. The world needs to move on.

what do you suggest as alternative? the problem is not linux, not mutable global state or resources and not libc. the problem is not getting time at work to do things properly. like spotting this in GDB before the issue hit, because your boss gave you time to tirelessly debug and reverse your code and anything it touches.... there is too much money in halfbaked code. sad but true.

It definitely is the current libc. That one's proven by systems which do not have the same problem. Then the next layer problem is trying to pretend we can get everyone to pay attention and avoid bugs in code instead of forcing interfaces and implementations where those bugs are not possible.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#74

Post by panzi »

vlovich123 wrote: post #29

Even if C stdlib maintainers are resistant against making setenv multi-thread safe, at a minimum there should be a new alternative thread-safe API defined, whether within POSIX or defining a defacto standard and forcing POSIX to adopt it over time. If instead of explaining why nothing could be done was spent fixing this problem, a new thread-safe API could have replaced the old setenv which could have been deprecated…

Guess that would also require some locking for all the exec() functions that don't take the environment as a parameter or that search PATH for the executable.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#75

Post by benatkin »

koito17 wrote: post #69
benatkin wrote: post #61

Earlier quoted context omitted.

React doesn't have a tag and attribute sanitizer built in, so having non-js-programmers edit JSX isn't especially safe anyways, as an img or a href could exfiltrate data. If it were they could just block out an innerHTML attribute. A js programmer can get around it by setting up a ref and then using the reference to set innerHTML without the word dangerously appearing.

> A js programmer can get around it by setting up a ref and then using the reference to set innerHTML without the word dangerously appearing. If DOM nodes during the next render differ from what react-dom expects (i.e. the DOM nodes from the previous render), then react-dom may throw a DOMException. Mutating innerHTML via a ref may violate React's invariants, and the library correctly throws an error when programmers…

The reference is used to operate on the subtree when wrapping libraries like CodeMirror https://github.com/uiwjs/react-codemirror/blob/master/core/s... React leaves it alone if the children doesn't change.

innerHTML is useful when there is a trusted HTML source, which is becoming more popular with stuff like HTMX and FastHTML.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#77

Post by usefulcat »

mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads.

It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#78

Post by kelnos »

StillBored wrote: post #39

Its like a rite of passage to be hit by an environment related bug on linux, which is mysteriously less a problem on other unix's. Which is sorta funny given how pragmatic Linus and the kernel are about fixing POSIX bugs by making them not happen, while glibc is still lagging here decades after people tried to at least make the problem better. Sure there is all the crap around TZ/etc, but simply providing getenv_r()…

> environment related bug on linux, which is mysteriously less a problem on other unix's.

How do you figure? The problem isn't the implementation, it's the API. setenv(), unsetenv(), putenv(), and especially environ, are inherently unsafe in a multithreaded program. Even getenv_r() can't really save you, since another thread may be calling setenv() while the (old) value of an env var is being copied into the provided buffer. Sure, a getenv_r() fixes the case where you get something back from getenv(), and then another thread calls setenv() and makes that memory invalid, but there's no way to protect the other calls breaking the API.

There are ways to mitigate some of the issues, like having libc hold a mutex when inside getenv()/setenv()/putenv()/unsetenv(), but there's still no way for libc to guarantee that something returned by getenv() remains valid long enough for the calling code to use it (which, right, can be fixed by getenv_r(), which could also be protected by that mutex). But there's no good way to make direct access to environ safe. I suppose you could make environ a thread-local, but then different threads' views of the environment could become out of sync, permanently (and you could get different results between calling getenv_r() and examining environ directly).

Back-compat here is just really hard to do. Even adding a mutex to protect those functions could change the semantics enough to break existing programs. (Arguably they're already broken in that case, but still...)

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#79

Post by panzi »

nomel wrote: post #71
kbolino wrote: post #62

Earlier quoted context omitted.

They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.

> and environment variables require an operating system Is that true? It's just a process global string -> string map, that can be pre-loaded with values before the process starts, with a copy of the current state being passed to any sub-process. This could be trivially implemented with batch processing/supervisory programs.

Well, it's used by the OS when exec-ing a new process, but at least the Linux syscall for that takes the environment as an explicit parameter. So it could be managed in whatever way by the runtime until execve() is called.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#80

Post by kelnos »

tsimionescu wrote: post #59
cogman10 wrote: post #41

Earlier quoted context omitted.

You can't. You could wrap setenv in a mutex, but that's not good enough. It can still be called from different processes, which means you'd need to do a more expensive and complex syncing system to make it safe. That ballons out to other env related methods needing to honor the synchronization primitive in order for there to be a semblance of safety. However, you still end up in a scenario where you can call setenv g…

Of course you can. Mutexes are system objects, so it's not a huge problem to sync across processes, if you really have to (is it really expected that one process can set env vars inside another process?). Making global state, especially state that has no reason to be modified or even read very often like the env, thread safe is a trivial issue, well studied and understood. Could an intern do it? Probably not. Could l…

That doesn't solve anything. You could be using a library (perhaps a closed-source one) that doesn't use these hypothetical lockenv()/unlockenv() functions.

This needs to be fixed inside libc, but there's no way to do so completely without breaking backward-compatibility.

Post reply on HN