Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

91–100 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#91

Post by PaulDavisThe1st »

dgrunwald wrote: post #37
umpalumpaaa wrote: post #25

Earlier quoted context omitted.

I am not sure making things safe by default is a good idea. This always comes with a cost. Thats also the reason why basic data types (array, dictionaries, etc) are generally not thread safe… because its usually not needed or handled on a much higher level. Its a different story for languages/environments that are supposed to be safe by default and where you have language features that ensure safety (actors, optional…

The problem with `setenv` is that people expect one process to have one set of environment variables, which is shared across multiple languages running in that process. This implies every language must let its environment variables be managed by a central language-independent library -- and on POSIX systems, that's libc. So if libc refuses to provide thread-safety, that impacts not just C, but all possible languages…

It's not just that "libc refuses to provide thread-safety" ... the POSIX standard specifies that these functions are non-reentrant.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#92

Post by pshc »

usefulcat wrote: post #77
mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

The underlying problem is that setenv is mutable global state and should never have existed

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#93

Post by ModernMech »

usefulcat wrote: post #77
mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

It's the same problem with global vars, but at a machine scope. The real solution here would be for the OS to have a better interface to read and write env vars, more like a file where you have to get rw permission (whether that's implemented as a mutex or what).

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#94

Post by sunshowers »

fsckboy wrote: post #45
Thaxll wrote: post #13

Earlier quoted context omitted.

Why requiring unsafe when the std implementation could take care of the synchronisation?

you've gotten a lot of answers which say the same thing, but which I don't think answer your question: synchronization methods impose various complexity and performance penalties, and single threaded applications which don't need that would pay those penalties and get no benefit. Unix was designed around a lightweight ethos that allowed simple combining of functions by the user on the command line. See "worse is bett…

Well it was better in the short term but is worse in the long term. In particular, the error handling situation is generally atrocious, which is fine for interactive/sysadmin use but much worse for serious production use.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#95

Post by jcotton42 »

miohtama wrote: post #86
masklinn wrote: post #21

Earlier quoted context omitted.

Because the std implementation can not force synchronisation on the libc, so any call into a C library which uses getenv will break... which is exactly what happened in TFA: `openssl-probe` called env::set_var on the Rust side, and the Python interpreter called getenv(3) directly.

Is it possible to skip libc completely or would this introduce too many portability concerns?

It's not just libc, it's any C or C++ library that calls getenv or setenv.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#96

Post by eqvinox »

usefulcat wrote: post #77
mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE.

And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be started with a different environment — build a completely new structure, don't touch the existing one. Same for posix_spawn.

And, lastly, compatibility with ancient systems strikes again: the environment is also accessible through this:

   extern char **environ;
Which is, of course, best described as bullshit.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#97

Post by PaulDavisThe1st »

kelnos wrote: post #82
plorkyeran wrote: post #55

Earlier quoted context omitted.

The problem is that applications sometimes need to set environment variables which will be read by libraries in the same process. This is safe to do during startup, but at no later times. Ideally all libraries which use environment variables should have APIs allowing you to override the env variables without calling setenv(), but that isn't always the case.

> The problem is that applications sometimes need to set environment variables which will be read by libraries in the same process. This is safe to do during startup, but at no later times. No, the problem is that libraries try to do this at all. Libraries should just have those APIs you mention, and not touch env vars, period. If you, the library user, really want to use env vars for those settings, you can getenv()…

If env vars don't exist, that makes it much harder (and more likely impossible) for users to modify library/application behavior at run time.

I agree with you that it would be much better if, when libA needs to set behavio Foo in libB, it called libB:setBehavior (Foo) rather than setenv ("LibBehavior", "Foo")

But let's not throw the baby out with the bathwater.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#98

Post by sunshowers »

nomel wrote: post #71
kbolino wrote: post #62

Earlier quoted context omitted.

They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.

> and environment variables require an operating system Is that true? It's just a process global string -> string map, that can be pre-loaded with values before the process starts, with a copy of the current state being passed to any sub-process. This could be trivially implemented with batch processing/supervisory programs.

Environment variables are not just technical, they're social. You need to get everyone on board with your scheme.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#99

Post by sunshowers »

IshKebab wrote: post #57
zanderwohl wrote: post #26

Earlier quoted context omitted.

It would be a tremendous amount of work, and would take years. Meanwhile, the problems are avoidable. It's not exactly the "rust way" to just remember and avoid problems, but everything in language design is compromises.

"Impossibru!!" https://github.com/sunfishcode/eyra Oh look: > Why use Eyra? It fixes Rust's set_var unsoundness issue. The environment-variable implementation leaks memory internally (it is optional, but enabled by default), so setenv etc. are thread-safe.

That only works on Linux though right?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#100

Post by eqvinox »

ModernMech wrote: post #93
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

It's the same problem with global vars, but at a machine scope. The real solution here would be for the OS to have a better interface to read and write env vars, more like a file where you have to get rw permission (whether that's implemented as a mutex or what).

This is neither an OS nor a machine scope problem. The environment is provided by the OS at startup. What the process does with it from there on is its own concern.
Post reply on HN