Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

101–110 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#101

Post by einpoklum »

kelnos wrote: post #78
StillBored wrote: post #39

Its like a rite of passage to be hit by an environment related bug on linux, which is mysteriously less a problem on other unix's. Which is sorta funny given how pragmatic Linus and the kernel are about fixing POSIX bugs by making them not happen, while glibc is still lagging here decades after people tried to at least make the problem better. Sure there is all the crap around TZ/etc, but simply providing getenv_r()…

> environment related bug on linux, which is mysteriously less a problem on other unix's. How do you figure? The problem isn't the implementation, it's the API. setenv(), unsetenv(), putenv(), and especially environ, are inherently unsafe in a multithreaded program. Even getenv_r() can't really save you, since another thread may be calling setenv() while the (old) value of an env var is being copied into the provided…

> Even getenv_r() can't really save you, since another thread may be calling setenv() while the (old) value of an env var is being copied into the provided buffer.

Won't that depends on the libc implementation. For example, maybe setenv writes to another buffer, then swaps pointers atomically; wouldn't that work?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#102

Post by sunshowers »

tsimionescu wrote: post #59
cogman10 wrote: post #41

Earlier quoted context omitted.

You can't. You could wrap setenv in a mutex, but that's not good enough. It can still be called from different processes, which means you'd need to do a more expensive and complex syncing system to make it safe. That ballons out to other env related methods needing to honor the synchronization primitive in order for there to be a semblance of safety. However, you still end up in a scenario where you can call setenv g…

Of course you can. Mutexes are system objects, so it's not a huge problem to sync across processes, if you really have to (is it really expected that one process can set env vars inside another process?). Making global state, especially state that has no reason to be modified or even read very often like the env, thread safe is a trivial issue, well studied and understood. Could an intern do it? Probably not. Could l…

That is a technical solution. What is your solution to the much more serious social problem of adding this check to every codebase in existence? What points of leverage do you have?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#103

Post by steveklabnik »

PaulDavisThe1st wrote: post #88
steveklabnik wrote: post #56

Earlier quoted context omitted.

Linux is an unusual platform in that it allows you to call into it via assembly. Most other platforms require you to go through libc to do so. It's not really in Rust's hands.

This is not unusual at all. Windows allowed it for years before Linux came along. It was also true of some other *nix systems - IIRC, Ultrix (DEC) allowed this, and so did Dynix (Sequent). *BSD allows it too, or used as of 2022. What is unusual about Linux is that it guarantees a syscall ABI, meaning that if you follow it, you can make a system call "portably" across "any" version of Linux.

Sure, I’m speaking about platforms that are relevant today, not historical ones. Windows, MacOS, {Free,Open,Net}BSD, Solaris, illumos, none of these do.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#104

Post by ModernMech »

eqvinox wrote: post #100
ModernMech wrote: post #93

Earlier quoted context omitted.

It's the same problem with global vars, but at a machine scope. The real solution here would be for the OS to have a better interface to read and write env vars, more like a file where you have to get rw permission (whether that's implemented as a mutex or what).

This is neither an OS nor a machine scope problem. The environment is provided by the OS at startup . What the process does with it from there on is its own concern.

> The environment is provided by the OS at startup.

That's part of the design of the OS. How the OS implements this is primitive, and so it leaves it up to every language to handle. The blog mentions the issue is with getenv, setenv, and realloc, all system calls. To me, that sounds like bad OS design is causing issues downstream with languages, leaving it up to individual programmers to deal with the fallout.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#105

Post by Joker_vD »

pshc wrote: post #92
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

The underlying problem is that setenv is mutable global state and should never have existed

The process's current directory is mutable global state as well, and yet chdir(2) is thread-safe.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#106

Post by einpoklum »

masklinn wrote: post #64
Animats wrote: post #43

Earlier quoted context omitted.

Yes. That's known. Most of the rest of the problem here seems to be the development environment. They're testing on a remote machine in an Amazon data center and using Docker. This rig fails to report that a process has crashed. Then they don't have enough debug symbol info inside their container to get a backtrace. If they'd gotten a clean backtrace reported on the first failure, this would have been obvious. Why is…

> Why is anyone using "setenv" anyway? Because it’s there and it looks like a good idea until it takes one of your fingers.

It really does not look like a good idea to setenv() . The very notion is quite terrifying. Messing with a bunch of globals, that other code knows about as well? Nuh-uh.

The thing is, the OP people weren't doing that at all, it was some irresponsible library maintainers. If your code does that, you have to include something like the "surgeon general's warning" everywhere: "CAREFUL: USING THIS LIBRARY MAY CAUSE TERMINAL CRASHES".

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#107

Post by diroussel »

eqvinox wrote: post #96
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

Indeed, environment variables should be used to configure child processes, not to configure the current process, for non-shell programs, IMHO.

Note that Java, and the JVM, doesn't allow changing environment variables. It was the right choice, even if painful at times.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#108

Post by duped »

miohtama wrote: post #86
masklinn wrote: post #21

Earlier quoted context omitted.

Because the std implementation can not force synchronisation on the libc, so any call into a C library which uses getenv will break... which is exactly what happened in TFA: `openssl-probe` called env::set_var on the Rust side, and the Python interpreter called getenv(3) directly.

Is it possible to skip libc completely or would this introduce too many portability concerns?

In general, no, because of FFI. In special circumstances, yes, but this isn't really important because the libc implementation is trivial (on all platforms that matter, envp is a char** to strings formatted as KEY=VALUE, set_env(key, value) is equivalent to allocating a new KEY=VALUE string and finding the index of a key if it exists or appending to the array).

Under the hood the pointer is initialized by the loader, in a special place in executable memory. Most of the time, the loader gets the initial environment variable list by looking at argv* (try reading past the end of the null separator, you'll find the initial environment variables).

It would be possible for a language to hack it such that on load they initialize their own env var set without using libc and be able to safely set/get those env vars without going through libc, and to inherit them when spawning child processes by reading the special location instead of the standard location initialized by your platforms' loader/updated by libc. But how useful is a language with FFI that's fundamentally broken since callees can't set environment variables? (probably very useful, since software that relies on this is questionably designed in the first place)

If you wanted to make a bullet proof solution, you would specify the location of an envp mutex in the loaders' format and make it libc's (or any language runtime) problem to acquire that mutex.

* there are platforms where this isn't true

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#109

Post by davidt84 »

fsckboy wrote: post #45
Thaxll wrote: post #13

Earlier quoted context omitted.

Why requiring unsafe when the std implementation could take care of the synchronisation?

you've gotten a lot of answers which say the same thing, but which I don't think answer your question: synchronization methods impose various complexity and performance penalties, and single threaded applications which don't need that would pay those penalties and get no benefit. Unix was designed around a lightweight ethos that allowed simple combining of functions by the user on the command line. See "worse is bett…

The real problem is that getenv() and setenv() were created before threads were really a thing.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#110

Post by Joker_vD »

eqvinox wrote: post #96
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv.

Mutating argv is actually quite popular, or at least it used to be.

Post reply on HN