Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

121–130 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#121

Post by wmf »

PaulDavisThe1st wrote: post #90
wmf wrote: post #14

Earlier quoted context omitted.

It's time to move beyond this attitude and make things safe by default. For example, Solaris has a safer version of setenv(). "It is ridiculous that this has been a known problem for so long. It has wasted thousands of hours of people's time, either debugging the problems, or debating what to do about it. We know how to fix the problem." https://www.evanjones.ca/setenv-is-not-thread-safe.html

One of the major differences between X Window and the win32 GUI APIs is that the windows one builds in thread safety, and it cannot be removed. This means that you pay the price of mutexes and the like (what the windows world likes to call "critical sections"), even if you have a single threaded GUI. X Window, on the other hand, decided to do nothing about threads at all, leaving it up to the application. 30 years af…

Yet 30 years later people are calling setenv()/getenv() from different threads even though "it is known" that it crashes. For whatever reason the lesson from GUIs doesn't apply here.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#122

Post by wmf »

cogman10 wrote: post #41
wmf wrote: post #14

Earlier quoted context omitted.

It's time to move beyond this attitude and make things safe by default. For example, Solaris has a safer version of setenv(). "It is ridiculous that this has been a known problem for so long. It has wasted thousands of hours of people's time, either debugging the problems, or debating what to do about it. We know how to fix the problem." https://www.evanjones.ca/setenv-is-not-thread-safe.html

You can't. You could wrap setenv in a mutex, but that's not good enough. It can still be called from different processes, which means you'd need to do a more expensive and complex syncing system to make it safe. That ballons out to other env related methods needing to honor the synchronization primitive in order for there to be a semblance of safety. However, you still end up in a scenario where you can call setenv g…

You didn't read the link, did you?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#123

Post by rerdavies »

kelnos wrote: post #78
StillBored wrote: post #39

Its like a rite of passage to be hit by an environment related bug on linux, which is mysteriously less a problem on other unix's. Which is sorta funny given how pragmatic Linus and the kernel are about fixing POSIX bugs by making them not happen, while glibc is still lagging here decades after people tried to at least make the problem better. Sure there is all the crap around TZ/etc, but simply providing getenv_r()…

> environment related bug on linux, which is mysteriously less a problem on other unix's. How do you figure? The problem isn't the implementation, it's the API. setenv(), unsetenv(), putenv(), and especially environ, are inherently unsafe in a multithreaded program. Even getenv_r() can't really save you, since another thread may be calling setenv() while the (old) value of an env var is being copied into the provided…

Why does adding a mutex break the API? I guess it breaks `char**environ`. But the API wouldn't be broken.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#124

Post by IshKebab »

kbolino wrote: post #63
IshKebab wrote: post #57

Earlier quoted context omitted.

"Impossibru!!" https://github.com/sunfishcode/eyra Oh look: > Why use Eyra? It fixes Rust's set_var unsoundness issue. The environment-variable implementation leaks memory internally (it is optional, but enabled by default), so setenv etc. are thread-safe.

That's quite a trade-off

What is? Leaking memory? It's going to be a few kB at absolute most. Not an issue unless you are doing something very weird.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#125

Post by HarHarVeryFunny »

What is the rationale for libc not making setenv/getenv thread safe? It does seem rather odd given how environment variables are explicitly defined as shared between threads in the same process!

It doesn't seem it would take much to do it efficiently, even retaining the poor getenv() pointer-returning API (which could point to a thread local buffer). The coordination between getenv and setenv could be very lightweight - spinlock vs mutex.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#126

Post by mmastrac »

kbolino wrote: post #63
IshKebab wrote: post #57

Earlier quoted context omitted.

"Impossibru!!" https://github.com/sunfishcode/eyra Oh look: > Why use Eyra? It fixes Rust's set_var unsoundness issue. The environment-variable implementation leaks memory internally (it is optional, but enabled by default), so setenv etc. are thread-safe.

That's quite a trade-off

I think glibc made the same trade-off. It makes sense for most types of programs, but there's certainly a lot of classes of programs that wouldn't take it.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#127

Post by gpm »

eqvinox wrote: post #119
gpm wrote: post #115

Earlier quoted context omitted.

> but argv is a function parameter to main()¹, and only that. > ¹ or technically whatever your ELF entry point is, _start in crt0 or your poison of choice. Once you include the footnote, at least on linux/macos (not sure about Windows), you could take the same perspective with regards to envp and the auxiliary array. It's libc that decided to store a pointer to these before calling your `main`, not the abi. At the ti…

I mean, yes, we're in "violent agreement" there. It's nice that libc squirrels away a copy and gives you a `getenv()` function with a string lookup, but… setenv… that was just a horrible idea. It's not really wrong to view it as a tool that allows you to muck around with main()'s local variables. Which to me sounds like one should take a shower after using it ;D (Ed.: the man page should say "you are required to take…

Oops, didn't mean to come across as disagreeing at all, more of a "yes, and ".

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#128

Post by josefx »

pshc wrote: post #92
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

The underlying problem is that setenv is mutable global state and should never have existed

Welcome to the C standard library, the application of mutable global state to literally everything in it has to be the most consistent and predictable feature of the language standard.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#129

Post by rikthevik »

Great article about digging into a non-obvious bug. This one had it all! Intermittent bug, architecture-specific, hidden in a dependency, rust, the python GIL, gettext. Fantastic stuff.

These kinds of detailed troubleshooting reports are the closest thing you can get to having to do it yourself. Thanks to the authors. It's easy to say "don't use X duh" until a dependency relies on it, and how were you supposed to know?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#130

Post by davidt84 »

vlovich123 wrote: post #29

Even if C stdlib maintainers are resistant against making setenv multi-thread safe, at a minimum there should be a new alternative thread-safe API defined, whether within POSIX or defining a defacto standard and forcing POSIX to adopt it over time. If instead of explaining why nothing could be done was spent fixing this problem, a new thread-safe API could have replaced the old setenv which could have been deprecated…

I'm not convinced by you that you know more than the experts who have determined there is no backwards-compatible way to fix this.
Post reply on HN