Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

191–200 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#191

Post by vrtx0 »

saagarjha wrote: post #186
vrtx0 wrote: post #182

Let me try to help: 1. If a process crashes and dumps, be sure to look at the system log of the cause (e.g. SIGSEGV, OOM, invalid instruction, etc.) 2. Be certain you’re looking at the right core dumps — I believe UID 1000 just means posix UserID (which is unrelated to a PID), though I don’t use containers. 3. Stay focused on the right level of abstraction — memory model details are great to know, but irrelevant here…

I'm kind of confused by this response. It doesn't seem to match the actual article? For example, they consulted the code to find what x20 had in it, rather than blindly guessing. Doing that is perfectly fine and even desirable when analyzing crashes. There is no forking mentioned. People call setenv all the time when trying to modify their own environment (hence the crashes!). Nobody said anything about the size of e…

x20 is a general purpose register; optimizing compilers can use it for any number of variables, immediate values or intermediate computations at different points within that same function — or none at all (the variable ep could be optimized away).

Re: fork(), I just meant to be thorough in explaining the environment is copied, not shared by processes. Setenv() only affects the process from which it’s called.

The array size bit in the article: The value 0x220 looks suspiciously close to the size of the old environment in 64-bit words (0x220 / 8 = 68), and this value was written over the terminating NULL of the environment block…

HTH!

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#192

Post by GoblinSlayer »

tsimionescu wrote: post #168
stouset wrote: post #166

Earlier quoted context omitted.

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. This holds for a lot of programs, but what if you're writing a shell?

Even then, you could maintain a separate copy of the environment that you control and freely mutate. Basically, during startup, you create a copy of the env you received. Any setenv primitive you expose to users will modify this copy (that you can sync properly yourself). When you want to launch a process, you explicitly provide the internal copy of the env to that process, you don't rely on libc providing its own co…

It doesn't look like there's any incentive to change it, e.g. getenv_r is an unpopular function.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#193

Post by rcxdude »

einpoklum wrote: post #106
masklinn wrote: post #64

Earlier quoted context omitted.

> Why is anyone using "setenv" anyway? Because it’s there and it looks like a good idea until it takes one of your fingers.

It really does not look like a good idea to setenv() . The very notion is quite terrifying. Messing with a bunch of globals, that other code knows about as well? Nuh-uh. The thing is, the OP people weren't doing that at all, it was some irresponsible library maintainers. If your code does that, you have to include something like the "surgeon general's warning" everywhere: "CAREFUL: USING THIS LIBRARY MAY CAUSE TERMIN…

Sadly, it's often the only way to adjust certain behaviors of certain libraries.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#194

Post by rcxdude »

dgrunwald wrote: post #32
shikon7 wrote: post #24

I wonder why it is so hard for Rust to implement its own safe stdlib independent of C.

How exactly would that help in this situation? If both Rust and C have independent standard libraries loaded into the same process, each would have an independent set of environment variables. So setting a variable from Rust wouldn't make it visible to the C code, which would break the article's usecase of configuring OpenSSL. The only real solution is to have the operating system provide a thread-safe way of managin…

If there was a libc implemented in rust (like https://github.com/redox-os/relibc), you could use that for the C code in the process, and you'd be sharing the relevant state.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#195

Post by saagarjha »

vrtx0 wrote: post #191
saagarjha wrote: post #186

Earlier quoted context omitted.

I'm kind of confused by this response. It doesn't seem to match the actual article? For example, they consulted the code to find what x20 had in it, rather than blindly guessing. Doing that is perfectly fine and even desirable when analyzing crashes. There is no forking mentioned. People call setenv all the time when trying to modify their own environment (hence the crashes!). Nobody said anything about the size of e…

x20 is a general purpose register; optimizing compilers can use it for any number of variables, immediate values or intermediate computations at different points within that same function — or none at all (the variable ep could be optimized away). Re: fork(), I just meant to be thorough in explaining the environment is copied, not shared by processes. Setenv() only affects the process from which it’s called. The arra…

No, it does not. I don't think you understand what you are talking about, because none of these actually address the points I brought up. They use the same words, but semantically they are talking about something completely different.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#196

Post by GoblinSlayer »

lamontcg wrote: post #159
eqvinox wrote: post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

Environment variables are a gigantic, decades-old hack that nobody should be using... but instead everyone has rejected file-based configuration management and everyone is abusing environment variables to inject config into "immutable" docker containers...

Everything already supports environment variables, and everyone and their dog have their own favorite yaml-based configuration management.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#197

Post by jeroenhd »

HarHarVeryFunny wrote: post #125

What is the rationale for libc not making setenv/getenv thread safe? It does seem rather odd given how environment variables are explicitly defined as shared between threads in the same process! It doesn't seem it would take much to do it efficiently, even retaining the poor getenv() pointer-returning API (which could point to a thread local buffer). The coordination between getenv and setenv could be very lightweigh…

The spec says it's not supposed to be thread safe.

There's also no real backwards compatible way of fixing setenv(). getenv() returns a pointer that can be read at any time, and then there's the *environment parameter that can also be used to read env variables.

IMO the entire API should be deprecated for a thread safe one, but until someone comes with a standard setenv() alternative that's implemented by the libc runtimes, we'll be stuck with the shitty POSIX API, and every year we will read blog posts about get/setenv() crashing processes.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#198

Post by Sharlin »

lamontcg wrote: post #159
eqvinox wrote: post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

Environment variables are a gigantic, decades-old hack that nobody should be using... but instead everyone has rejected file-based configuration management and everyone is abusing environment variables to inject config into "immutable" docker containers...

It's funny how any hack, no matter how big, somehow becomes a commonplace everyday "solution" once it's needed to work around some quirk of whatever technology is fashionable at the time.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#199

Post by Sharlin »

josefx wrote: post #128
pshc wrote: post #92

Earlier quoted context omitted.

The underlying problem is that setenv is mutable global state and should never have existed

Welcome to the C standard library, the application of mutable global state to literally everything in it has to be the most consistent and predictable feature of the language standard.

I mean, I'm sure it was an okay solution on PDP-11.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#200

Post by saagarjha »

HarHarVeryFunny wrote: post #125

What is the rationale for libc not making setenv/getenv thread safe? It does seem rather odd given how environment variables are explicitly defined as shared between threads in the same process! It doesn't seem it would take much to do it efficiently, even retaining the poor getenv() pointer-returning API (which could point to a thread local buffer). The coordination between getenv and setenv could be very lightweigh…

The rationale is that it was implemented before threads existed, and now can't be retrofitted with thread safety.
Post reply on HN