Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

141–150 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#141

Post by steveklabnik »

PaulDavisThe1st wrote: post #140
steveklabnik wrote: post #138

Earlier quoted context omitted.

In my understanding, Go initially disregarded various platforms' rules here, and have ended up walking it back. I could be wrong though. It's hard to find good details here, but here's a mailing list thread from 2019 mentioning libc usage: https://groups.google.com/g/golang-nuts/c/uX8eUeyuuAY/m/Cfhl... > On Solaris (and Windows), and more recently in macOS as well we link with libc (or equivalent). > Go used to do ra…

Yep, in 2022 it finally started using libc on *BSD too. But ... there's a difference between being able to do direct syscalls via asm, and them being portable across kernel versions, which is what this subthread was about. Granted, most people want version portability, but still on a technical level, it's not the same thing.

No, my comment was about what APIs a platform considers to be their stable, external API. That you can technically call them anyway (except for ones like OpenBSD that actively check and prevent you) doesn't mean you're not doing something unsupported.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#142

Post by benmmurphy »

rerdavies wrote: post #123
kelnos wrote: post #78

Earlier quoted context omitted.

> environment related bug on linux, which is mysteriously less a problem on other unix's. How do you figure? The problem isn't the implementation, it's the API. setenv(), unsetenv(), putenv(), and especially environ, are inherently unsafe in a multithreaded program. Even getenv_r() can't really save you, since another thread may be calling setenv() while the (old) value of an env var is being copied into the provided…

Why does adding a mutex break the API? I guess it breaks `char**environ`. But the API wouldn't be broken.

I think you would have to change the API to return a copy of the string as the get_env result which the caller is responsible for free-ing or the env implementation would have to ensure returned values from get_env are stable and never change which is effectively a memory leak.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#143

Post by josefx »

debugnik wrote: post #132
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

No amount of locking can make the getenv API thread-safe, because it returns a pointer which gets invalidated by setenv, but lacks a way to release ownership over it and unblock setenv safely (or to free a returned copy). So setenv's existence makes getenv inherently unsafe unless you can ensure the entire application is at a safe point to use them.

C could provide functions to lock/unlock a mutex and require that any attempt to access the environment has to be done holding the mutex. This would still leave the correctness in the hands of the user, but at least it would provide a standard API to secure the environment in a multi threaded application that library and application developers could adopt.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#144

Post by jamesfinlayson »

diroussel wrote: post #107
eqvinox wrote: post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

Indeed, environment variables should be used to configure child processes, not to configure the current process, for non-shell programs, IMHO. Note that Java, and the JVM, doesn't allow changing environment variables. It was the right choice, even if painful at times.

Sure is painful (mostly when writing tests where the environment variables aren't abstracted in some way).

But I think it was actually possible to hack around up until Java 17.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#145

Post by plorkyeran »

kelnos wrote: post #82
plorkyeran wrote: post #55

Earlier quoted context omitted.

The problem is that applications sometimes need to set environment variables which will be read by libraries in the same process. This is safe to do during startup, but at no later times. Ideally all libraries which use environment variables should have APIs allowing you to override the env variables without calling setenv(), but that isn't always the case.

> The problem is that applications sometimes need to set environment variables which will be read by libraries in the same process. This is safe to do during startup, but at no later times. No, the problem is that libraries try to do this at all. Libraries should just have those APIs you mention, and not touch env vars, period. If you, the library user, really want to use env vars for those settings, you can getenv()…

The place where it makes sense for a library to read environment variables is where the program is not written to use that specific library. For example, I can link a program whose author has never heard of TCMalloc against TCMalloc rather than the system malloc, and then configure TCMalloc via environment variables. This does not require modifying a single line of code, while manually forwarding configuration onto the allocator would. Another common example is configuring sanitizers. Not having to do anything other than pass another command-line switch to the compiler is one of the things that makes them really painless to use.

I do think you'd be hard-pressed to find a situation where a program calling setenv() to configure a library actually makes sense. It's a pretty strong sign that someone made a bad decision. People will, however, make mistakes in API design.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#146

Post by forrestthewoods »

01HNNWZ0MV43FF wrote: post #6
forrestthewoods wrote: post #3

Mutable global state is evil. Friends don’t let friends use mutable global state. I hate envvars. It’s “the Linux way”. I avoid them like the plague. A++ strong recommend. libc is terrible. The world needs to move on.

Env vars are good if you treat them as read-only within the process

I’ll take a config file over an envvar 100% of the time.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#147

Post by vlovich123 »

davidt84 wrote: post #130
vlovich123 wrote: post #29

Even if C stdlib maintainers are resistant against making setenv multi-thread safe, at a minimum there should be a new alternative thread-safe API defined, whether within POSIX or defining a defacto standard and forcing POSIX to adopt it over time. If instead of explaining why nothing could be done was spent fixing this problem, a new thread-safe API could have replaced the old setenv which could have been deprecated…

I'm not convinced by you that you know more than the experts who have determined there is no backwards-compatible way to fix this.

I'll take existence proofs [1] over personal insults but YMMV. You also may want to be careful assuming the expertise of people on this forum. Some people here are quite technical.

[1] https://github.com/bminor/glibc/commit/7a61e7f557a97ab597d6f...

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#148

Post by vlovich123 »

usefulcat wrote: post #113
vlovich123 wrote: post #29

Even if C stdlib maintainers are resistant against making setenv multi-thread safe, at a minimum there should be a new alternative thread-safe API defined, whether within POSIX or defining a defacto standard and forcing POSIX to adopt it over time. If instead of explaining why nothing could be done was spent fixing this problem, a new thread-safe API could have replaced the old setenv which could have been deprecated…

The biggest problem is not the absence of a thread safe API, it's the existence of this: extern char **environ; As long as environ is publicly accessible, there's no guarantee that setenv and getenv will be used at all, since they're not necessary. If you're willing to get rid of environ, it's pretty trivial to make setenv and getenv thread safe. If not, then it's impossible, although one could still argue that makin…

> aka don't let the perfect be the enemy of the good

Exactly my point. Over time *environ would disappear, at least from the major software projects that everyone uses (assuming it's even in use in them in the first place).

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#149

Post by davidt84 »

vlovich123 wrote: post #147
davidt84 wrote: post #130

Earlier quoted context omitted.

I'm not convinced by you that you know more than the experts who have determined there is no backwards-compatible way to fix this.

I'll take existence proofs [1] over personal insults but YMMV. You also may want to be careful assuming the expertise of people on this forum. Some people here are quite technical. [1] https://github.com/bminor/glibc/commit/7a61e7f557a97ab597d6f...

That isn't thread safe, it's safER.

I am also quite technical, thanks.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#150

Post by gavinhoward »

mmastrac wrote: post #54
gavinhoward wrote: post #49

It is weird that I got this right before Rust did. Because I use structured concurrency, I can make it so every thread has its own environment stack . To add to a new environment, I duplicate it, add the new variable, and push the new enviroment on the stack. Then I can use code blocks to delimit where that stack should be popped. [1] This is all perfectly safe, no `unsafe` required, and can even extend to other thin…

This isn't _really_ a Rust problem. Rust is a victim of POSIX. If you have 1) C FFI interop in Yao, there's still a chance you might have two C libraries cause a crash without your code even being involved.

Except if there is dymanic linking, I can use that to inject my own setenv and getenv, just like people inject jemalloc or other malloc alternatives.
Post reply on HN