Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

151–160 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#151

Post by GuB-42 »

Joker_vD wrote: post #110
eqvinox wrote: post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. Mutating argv is actually quite popular, or at least it used to be.

Mutating argv is fine for how it is usually done. That is, to permute the arguments in a getopt() call so that all nonoptions are at the end.

It is fine because it is usually done during the initialization phase, before starting any other thread. setenv() can be used here too, though I prefer to avoid doing that in any case. I also prefer not to touch argv, but since that's how GNU getopt() works, I just go with it.

Once the program is running and has started its threads, I consider setenv() is a big no no. The Rust documentation agrees with me: "In multi-threaded programs on other operating systems, the only safe option is to not use set_var or remove_var at all.". Note: here, "other operating systems" means "not Windows".

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#152

Post by hinkley »

diroussel wrote: post #107
eqvinox wrote: post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

Indeed, environment variables should be used to configure child processes, not to configure the current process, for non-shell programs, IMHO. Note that Java, and the JVM, doesn't allow changing environment variables. It was the right choice, even if painful at times.

I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.

I am fairly certain that somewhere inside the polyhedron that satisfies those constraints, is a large subset that could be statically analyzed and proven sound. But I'm less certain if Rust could express it cleanly.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#153

Post by demurgos »

rerdavies wrote: post #118
masklinn wrote: post #21

Earlier quoted context omitted.

Because the std implementation can not force synchronisation on the libc, so any call into a C library which uses getenv will break... which is exactly what happened in TFA: `openssl-probe` called env::set_var on the Rust side, and the Python interpreter called getenv(3) directly.

But the standard implementation could copy the environment at startup, and only uses its copy. And the library's use of setenv is clearly a bug as setenv is documented to be not threadsafe in the C standard library. So that would take care of that problem.

If you clone the environment at startup, then you get a situation where code in the same binary can see different values depending if it uses libc or Rust's std. It's also no longer the same environment as in the process metadata.

Using a copy by default may have worked if it was designed as such before Rust 1.0, but Rust took the decision to expose the real environment and changing this now would be more disruptive than marking mutations as unsafe.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#154

Post by incrudible »

maep wrote: post #12
forrestthewoods wrote: post #3

Mutable global state is evil. Friends don’t let friends use mutable global state. I hate envvars. It’s “the Linux way”. I avoid them like the plague. A++ strong recommend. libc is terrible. The world needs to move on.

> Mutable global state is evil. Friends don’t let friends use mutable global state. Throw away your CPU and RAM then.

Your CPU has an MMU in order to (among other things) let the OS prevent mutable global state.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#155

Post by incrudible »

titzer wrote: post #17
maep wrote: post #12

Earlier quoted context omitted.

> Mutable global state is evil. Friends don’t let friends use mutable global state. Throw away your CPU and RAM then.

And disks. And the cloud. Or basically, you know, computers.

Ah yes, the cloud where we all happily share compute resources without any restrictions to avoid stomping on each others toes.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#156

Post by fanf2 »

plorkyeran wrote: post #139
Joker_vD wrote: post #105

Earlier quoted context omitted.

The process's current directory is mutable global state as well, and yet chdir(2) is thread-safe.

chdir is thread-safe, but interacting with the current directory in any context other than parsing command-line arguments is still nearly always a mistake. Everything past a program's entry point should be working exclusively in absolute paths.

Yeah if you chdir() in a multithreaded program, all cwd-relative file accesses in other threads are fucked.

As well as absolute paths, it’s ok to work with descriptor-relative paths using openat() and friends.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#157

Post by __MatrixMan__ »

kelnos wrote: post #85

This reminded me of that whole "12-factor app" movement, which several of my former coworkers had really bought into. One of the "factors" is that apps should be configured by environment variables. I always thought this was kinda foolish: your configuration method is a flat-namespace basked of stringly-typed values. The perils of getenv()/setenv()/environ are also, I think, a great argument against using env vars fo…

I have similar reservations about env vars. I dislike how they can be read from anywhere--it interrupts the ability to reason about a function's behavior from its signature and makes impure plenty of functions that could otherwise have been pure.

If there were a language feature that let me mark apps such that during any process env vars are not writable and are readable only once (together, in a batch, not once per var), I'd use it everywhere.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#158

Post by subarctic »

01HNNWZ0MV43FF wrote: post #8
jandrese wrote: post #2

Yet another person is burned by calling setenv() in a multi-threaded context. There really needs to be a big warning banner on the manpage for setenv() that warns about this because it seems like a far more common problem than you would expect.

Funny enough, the Rust wrapper `std::env::set_var` does have a big warning https://doc.rust-lang.org/std/env/fn.set_var.html

Looks like that Safety section was added in 1.76.0. It'll be an even bigger warning in the future since it's now going to be unsafe in Rust 2024

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#159

Post by lamontcg »

eqvinox wrote: post #96
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

Environment variables are a gigantic, decades-old hack that nobody should be using... but instead everyone has rejected file-based configuration management and everyone is abusing environment variables to inject config into "immutable" docker containers...

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#160

Post by nomel »

kbolino wrote: post #84
nomel wrote: post #71

Earlier quoted context omitted.

> and environment variables require an operating system Is that true? It's just a process global string -> string map, that can be pre-loaded with values before the process starts, with a copy of the current state being passed to any sub-process. This could be trivially implemented with batch processing/supervisory programs.

Sure, there's a broader concept here, which doesn't require any operating system. But any alternate string->string map you define won't answer to C code calling getenv, won't be passed to child processes created with fork, won't be visible through /proc/$PID/environ, etc.

This is the context:

> They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.

I think there's some confusion here. The C standard library is an abstraction layer that exists to implement standard behavior on hardware. It's entirely unrelated to the existence of an OS. Things like "/proc/$PID/environ" have nothing to do with C.

There are many standard libraries, for embedded, that implement these things, like getenv, on bare metal [1].

Standard C libraries exist to implement functionality. It does not define how to implement the functionality. That's the whole point of C: it's an abstraction that has very little requirements.

The implementation of environment variables don't require an OS. If they made this "core", they could trivially implement the concept.

[1] https://en.wikipedia.org/wiki/Newlib [2] getenv: https://sourceware.org/newlib/libc.html

Post reply on HN