Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

161–170 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#161

Earlier quoted context omitted.

> Why is anyone using "setenv" anyway? Because it’s there and it looks like a good idea until it takes one of your fingers.

It really does not look like a good idea to setenv() . The very notion is quite terrifying. Messing with a bunch of globals, that other code knows about as well? Nuh-uh. The thing is, the OP people weren't doing that at all, it was some irresponsible library maintainers. If your code does that, you have to include something like the "surgeon general's warning" everywhere: "CAREFUL: USING THIS LIBRARY MAY CAUSE TERMIN…

It's OpenSSL. It's basically a sea urchin turned into code in terms of safe handling.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#162

Earlier quoted context omitted.

Who has a signal safe malloc?

POSIX does not require malloc to be signal safe. Therefore I do not think that anyone has bothered to implement a signal-safe malloc, as this is likely to be complicated. Allocating memory in a signal handler makes no sense in a well designed program, so not being allowed to use malloc and related functions is not a problem.

So it sounds like malloc would be an example of a function which is thread safe but not reentrant.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#163
post #80

Earlier quoted context omitted.

Of course you can. Mutexes are system objects, so it's not a huge problem to sync across processes, if you really have to (is it really expected that one process can set env vars inside another process?). Making global state, especially state that has no reason to be modified or even read very often like the env, thread safe is a trivial issue, well studied and understood. Could an intern do it? Probably not. Could l…

That doesn't solve anything. You could be using a library (perhaps a closed-source one) that doesn't use these hypothetical lockenv()/unlockenv() functions. This needs to be fixed inside libc, but there's no way to do so completely without breaking backward-compatibility.

Yes, I was talking about fixes inside libc. The poster above was claiming it can't be done inside libc. And the lockvenv/unlockvenv functions I was mentioning were meant to exist besides the internal locking inside setenv/getenv. They would only be used if you needed transactional access (a combination of setting/getting multiple env vars atomically).

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#164

Earlier quoted context omitted.

> Why is anyone using "setenv" anyway? Because it’s there and it looks like a good idea until it takes one of your fingers.

It really does not look like a good idea to setenv() . The very notion is quite terrifying. Messing with a bunch of globals, that other code knows about as well? Nuh-uh. The thing is, the OP people weren't doing that at all, it was some irresponsible library maintainers. If your code does that, you have to include something like the "surgeon general's warning" everywhere: "CAREFUL: USING THIS LIBRARY MAY CAUSE TERMIN…

Agreed.

History: V7 research UNIX had "getenv()", but not "setenv()".[1] BSD Unix 4.x had "getenv()" and "setenv()"[2] Google's "AI Overview" says "The setenv() and unsetenv() functions were included in Version 7 of AT&T UNIX.", but that does not seem to be correct.

This misfeature seems to be what was once called a "Berkeleyism", a Berkeley mod to UNIX.

"setenv()" predates UNIX/Linux getting threads.

[1] http://web.cuzuco.com/~cuzuco/v7/v7vol1.pdf

[2] https://archive.org/details/44bsdprogrammers0000ucbe/page/n3...

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#165

Earlier quoted context omitted.

Of course you can. Mutexes are system objects, so it's not a huge problem to sync across processes, if you really have to (is it really expected that one process can set env vars inside another process?). Making global state, especially state that has no reason to be modified or even read very often like the env, thread safe is a trivial issue, well studied and understood. Could an intern do it? Probably not. Could l…

That is a technical solution. What is your solution to the much more serious social problem of adding this check to every codebase in existence? What points of leverage do you have?

The point was about adding a mutex inside libc in getenv and setenv. That way, every codebase in existence automatically gets this safety. The poster I was replying to claimed that this wouldn't help, because it would still not offer thread safety when doing multiple operations.

I pointed out that, in addition to libc setenv/getenv using a mutex internally, they could also expose new functions to allow transactional access for anyone that really needs it - though I suspect that is a vanishingly small minority.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#166
post #96

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv.

This holds for a lot of programs, but what if you're writing a shell?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#167

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

No amount of locking can make the getenv API thread-safe, because it returns a pointer which gets invalidated by setenv, but lacks a way to release ownership over it and unblock setenv safely (or to free a returned copy). So setenv's existence makes getenv inherently unsafe unless you can ensure the entire application is at a safe point to use them.

This is actually not that hard to fix.

Getenv() could keep several copies of the value around: one internal copy protected by a mutex, that it never returns, and one copy per thread that it stores in thread local storage. When you call getenv(), it locks the mutex, checks if the current thread's value exists, populates it from the internal copy if not, and returns it. It will also install a new setenv-specific signal handler on this thread and store info about this thread having a copy.

Setenv() will then take the same mutex as getenv(), check if the internal copy is different from the new value; if it is, it will modify the internal copy, modify the local thread's copy if that has one, and then signal each other thread in the process that has a copy in TLS. The setenv signal handler will modify the local copy that thread holds.

It's gonna be slow for a large multi-threaded program, but since setenv() used to corrupt memory for such programs, they probably don't care. And for single-threaded programs, or even for programs that don't access getenv()/setenv() on multiple threads, there should be no extra overhead other than the mutex and the bookkeeping.

The only issues that would remain are programs which send the pointer they get from getenv() to other threads without ensuring locking access, and programs which rely on modifying the pointer from getenv() directly as a way to set an env var, and expect this to be visible across threads. Those are just hopelessly broken and can't use the same API - but aren't more broken then they are today.

Of course, in addition to this complex work to make the old API (mostly) thread safe, it should also offer a new API that simply returns a copy every time, doesn't promise to show modifications to your copy when setenv() gets called (you need to call getenv() again), and puts the onus on you to free that copy explicitly.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#168
post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. This holds for a lot of programs, but what if you're writing a shell?

Even then, you could maintain a separate copy of the environment that you control and freely mutate. Basically, during startup, you create a copy of the env you received. Any setenv primitive you expose to users will modify this copy (that you can sync properly yourself). When you want to launch a process, you explicitly provide the internal copy of the env to that process, you don't rely on libc providing its own copy.

Of course, this means you won't see any changes to env vars from libraries you may use that call setenv(), but you also shouldn't need, or want, that in a shell.

I still think having a proper synchronous thread safe setenv()/getenv() in libc is the better choice.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#169
post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. This holds for a lot of programs, but what if you're writing a shell?

If you're writing a shell, you can spend the 15 minutes to write a custom mutable data structure for your envvars; no need to significantly worsen the entire ecosystem to reduce the size of shells by a couple dozen lines (or, rather, move those lines into libc..)

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#170
post #85

This reminded me of that whole "12-factor app" movement, which several of my former coworkers had really bought into. One of the "factors" is that apps should be configured by environment variables. I always thought this was kinda foolish: your configuration method is a flat-namespace basked of stringly-typed values. The perils of getenv()/setenv()/environ are also, I think, a great argument against using env vars fo…

This is unrelated really. If you read your enviornment variables into config and never touched them again, then you're totally safe.

I personally use 12 factor app style, but once it's entered the app I validate the env variables and data and then store them. It's totally fine after that.

Post reply on HN