Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

181–190 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#181

Post by 4gotunameagain »

HarHarVeryFunny wrote: post #125

What is the rationale for libc not making setenv/getenv thread safe? It does seem rather odd given how environment variables are explicitly defined as shared between threads in the same process! It doesn't seem it would take much to do it efficiently, even retaining the poor getenv() pointer-returning API (which could point to a thread local buffer). The coordination between getenv and setenv could be very lightweigh…

I think the argument was that the standard states that setenv is not thread safe, although from what I see it says that it does not have to be thread safe:

  The setenv( ) function need not be thread-safe. A function that is not required to be thread-safe is not required to be reentrant.
https://www.open-std.org/jtc1/sc22/open/n4217.pdf.

Page.. 1860 :')

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#182

Post by vrtx0 »

Let me try to help:

1. If a process crashes and dumps, be sure to look at the system log of the cause (e.g. SIGSEGV, OOM, invalid instruction, etc.)

2. Be certain you’re looking at the right core dumps — I believe UID 1000 just means posix UserID (which is unrelated to a PID), though I don’t use containers.

3. Stay focused on the right level of abstraction — memory model details are great to know, but irrelevant here.

4. Variables do not correlate 1:1 with registers, except in C calling conventions. The assumption about x20 and a local variable is incorrect, unfortunately.

5. getenv() and setenv() do not work as implied in the post. When a process starts via execve(), the OS/libc constructs a new snapshot of the environment, and cannot be modified by an ancestral process. It’s a snapshot in time, unless updated by the process itself. When a process fork()s, the child gets a new copy of the parent’s environment — updates do not propagate.

getenv() is thread safe and reentrant. You don’t use an environment to pass shared data — setenv() is generally used when constructing the environment for a child process before a fork(). See man environment.

6. FWIW, ‘char** env’ is a null-terminated array of pointers, so dumping memory from *env (or env[0]) is only valid until you hit the first NULL. The size of the array is not stored in the array.

I hope this helps! And apologies if this is redundant — I read so many comments; mostly variations of “the problem with getenv is x”, but gave up before reading all of the (currently) 168 comments.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#183

Post by sim7c00 »

viraptor wrote: post #73
sim7c00 wrote: post #23

Earlier quoted context omitted.

what do you suggest as alternative? the problem is not linux, not mutable global state or resources and not libc. the problem is not getting time at work to do things properly. like spotting this in GDB before the issue hit, because your boss gave you time to tirelessly debug and reverse your code and anything it touches.... there is too much money in halfbaked code. sad but true.

It definitely is the current libc. That one's proven by systems which do not have the same problem. Then the next layer problem is trying to pretend we can get everyone to pay attention and avoid bugs in code instead of forcing interfaces and implementations where those bugs are not possible.

just because someone makes a window doesn't mean you gotta jump out of it. there are good and bad uses for things, and the bad ones should be avoided lest one hurt themselves?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#184

Post by jabl »

adrian_b wrote: post #48
jabl wrote: post #34

Earlier quoted context omitted.

> A non-reentrant function cannot be thread safe. Actually, a non-reentrant function can be thread-safe. A common example of such a function in libc being malloc().

By definition, a "reentrant function" is a function that may be invoked even when it has not returned yet from a previous invocation. So a non-reentrant function is a function that may not be invoked again between a previous invocation and returning from that invocation. When a function may be invoked from different threads, then it is certain that sometimes it will be invoked by a thread before returning from a prev…

No, this is confused. Reentrancy ("reentrant-safe", or the somewhat related POSIX definition of async-signal-safe) and thread safety are not the same thing.

A reentrant function is thread-safe, but a thread-safe function may or may not be reentrant.

For instance, if a function uses mutual exclusion (say, posix_mutex_lock() and friends) to ensure thread-safety it won't be reentrant, because if the function is invoked via a signal handler it may deadlock. Which is why many common libc functions like malloc and stdio are not required to be async-signal-safe in POSIX, whereas they are required to be thread-safe.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#185

Post by viraptor »

sim7c00 wrote: post #183
viraptor wrote: post #73

Earlier quoted context omitted.

It definitely is the current libc. That one's proven by systems which do not have the same problem. Then the next layer problem is trying to pretend we can get everyone to pay attention and avoid bugs in code instead of forcing interfaces and implementations where those bugs are not possible.

just because someone makes a window doesn't mean you gotta jump out of it. there are good and bad uses for things, and the bad ones should be avoided lest one hurt themselves?

https://en.wikipedia.org/wiki/Death_of_Garry_Hoy people will assume more safety than necessary. You don't have to jump, but someone will try. We can accept that fact or watch people fail over and over on the same issue. It's better to help everyone avoid the problem in the first place.

For some reason lots of programmers will behave like the comment section on an accident video. "I would notice that earlier", "I'd avoid that", "I can react faster".

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#186

Post by saagarjha »

vrtx0 wrote: post #182

Let me try to help: 1. If a process crashes and dumps, be sure to look at the system log of the cause (e.g. SIGSEGV, OOM, invalid instruction, etc.) 2. Be certain you’re looking at the right core dumps — I believe UID 1000 just means posix UserID (which is unrelated to a PID), though I don’t use containers. 3. Stay focused on the right level of abstraction — memory model details are great to know, but irrelevant here…

I'm kind of confused by this response. It doesn't seem to match the actual article? For example, they consulted the code to find what x20 had in it, rather than blindly guessing. Doing that is perfectly fine and even desirable when analyzing crashes. There is no forking mentioned. People call setenv all the time when trying to modify their own environment (hence the crashes!). Nobody said anything about the size of env.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#187

Post by saagarjha »

dietr1ch wrote: post #172
hinkley wrote: post #152

Earlier quoted context omitted.

I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.…

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

Everyone thinks they are can be the first to do something, and that there is surely nothing that will happen before them. Unfortunately everyone save for one is mistaken. Sometimes that chosen one is not even consistent.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#188

Post by GoblinSlayer »

dietr1ch wrote: post #172
hinkley wrote: post #152

Earlier quoted context omitted.

I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.…

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

grpc reads some configuration from environment; environment has portability problems too, so it's useful to set it to cross platform shape.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#189

Post by friendzis »

dietr1ch wrote: post #172
hinkley wrote: post #152

Earlier quoted context omitted.

I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.…

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

`main` is the default entrypoint, with one simple argument to the linker you can change entrypoint symbol to whatever you wish.

You can add `premain` function that calls `main` and set it as an entrypoint, you can implement pre-start logic in main and call main loop later.

This is how any sane program is written anyway: set up environment -> continue with business logic

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#190

Post by saagarjha »

GuB-42 wrote: post #151
Joker_vD wrote: post #110

Earlier quoted context omitted.

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. Mutating argv is actually quite popular, or at least it used to be.

Mutating argv is fine for how it is usually done. That is, to permute the arguments in a getopt() call so that all nonoptions are at the end. It is fine because it is usually done during the initialization phase, before starting any other thread. setenv() can be used here too, though I prefer to avoid doing that in any case. I also prefer not to touch argv, but since that's how GNU getopt() works, I just go with it.…

A big reason to mutate argv is to change the process's name for tools like top.
Post reply on HN