Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

351–360 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#351

Post by robertlagrant »

dietr1ch wrote: post #172
hinkley wrote: post #152

Earlier quoted context omitted.

I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.…

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

> When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things

If you're only reading environment variables you have no problem, though. It's only if you try to change them that it causes issues.

For setting, "only set environment variables in the Bash script that starts your program" might be a good rule.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#352

Post by dietr1ch »

friendzis wrote: post #189
dietr1ch wrote: post #172

Earlier quoted context omitted.

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

`main` is the default entrypoint, with one simple argument to the linker you can change entrypoint symbol to whatever you wish. You can add `premain` function that calls `main` and set it as an entrypoint, you can implement pre-start logic in main and call main loop later. This is how any sane program is written anyway: set up environment -> continue with business logic

I know I can fool around with crt0, but I'm not sure how much you can really use that if you plan to use libraries that may depend on global `static` things that get created as they are linked in before `main` starts.

Maybe it's possible, but if I need to review every library (and hope they don't break my assumptions later) I think I lost on building this separation in practical way.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#353

Post by HarHarVeryFunny »

debugnik wrote: post #328
HarHarVeryFunny wrote: post #281

Earlier quoted context omitted.

Sure, but given that Linux defines the environment as state that's shared between threads, not having a thread-safe way of accessing it is hard to defend... Is "the standard says it doesn't NEED to be thread safe" the argument that the Linux libc maintainers are using for not enhancing it to be thread safe, or is it based on some technical or backwards compatibility issues in doing so ?

The only thread-safe way to implement getenv/setenv as they currently exist is to leak the previous state when setenv allocates, such that existing pointers stay valid. The existing API simply lacks a mechanism to synchronize correctly. Leaking would be good enough for many use cases, but it would break long-running users of setenv (mainly those with libraries abusing env vars, as in TFA), and doesn't even solve how…

Do you mean pointers returned by getenv() ? Those could point to thread-local buffers that data gets copied into when getenv() is called.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#354

Post by kazinator »

davidt84 wrote: post #350
kazinator wrote: post #342

Earlier quoted context omitted.

getenv can easily be misused in a single threaded program.

But it is possible to safely use it in a single threaded program. There's no way to use it safely in a multi threaded application that may use setenv (unless you add your own synchronisation, and ensure everything uses it, even third party libraries).

Actually I don't believe that's the case. The getenv function as described by ISO C cannot be safely used in a program that only uses getenv, if that program uses ISO C threads, and more than one threat calls getenv without synchronizing with the others.

I don't think POSIX fixes this: it doesn't specify that the environ array is protected against concurrent access.

If two threads call getenv right around the same time, one of them could invalidate the environ array just as the other one has started to traverse it.

If you want to be safe, copy the environment to a different data structure on program startup. Then have all your threads refer to that data structure.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#355

Post by GoblinSlayer »

gpderetta wrote: post #253
fch42 wrote: post #218

Earlier quoted context omitted.

The "cross platform" way of setting the environment is to set it "from outside" of the program - meaning, through the executor, whether that's the shell or the container runtime or even the kernel commandline if you insist to rewrite init in rust/go/zig/... It can be as-easy-as spawning your process via "env -i VAR1=... ... myprogram ..." - and given this also clears the dangers of env-insertion exploits, it's good p…

At the limit a program can execve itself with the new env.

It's not cross platform. Does java provide such interface?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#356

Post by GoblinSlayer »

gobblegobble2 wrote: post #247
GoblinSlayer wrote: post #206

Earlier quoted context omitted.

For that you write to /proc/self/comm, that's where top gets it from.

It may work for top, but not ps among others. The only reliable way is clobbering argv. That's just the way it is. In my opinion, glibc should finally provide setproctitle(), so programs like postgresql or chrome ( https://source.chromium.org/chromium/chromium/src/+/main:bas... ) don't have to resort to argv hacks.

There's prctl https://man7.org/linux/man-pages/man2/PR_SET_NAME.2const.htm... that writes to /proc/self/comm

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#357

Post by acuozzo »

nyrikki wrote: post #324
acuozzo wrote: post #270

Earlier quoted context omitted.

> instead everyone has rejected file-based configuration management With good reason. Files are surprisingly hard: https://danluu.com/deconstruct-files/

Rejecting one hard problem and replacing it with another method that is officially documented to be worse isn't really a solution. Note the standard: https://pubs.opengroup.org/onlinepubs/009604499/functions/se... > The setenv() function need not be reentrant. A function that is not required to be reentrant is not required to be thread-safe. With the increased use of PIE, thunks for both security and due to ARM + the…

> replacing it with another method that is officially documented to be worse isn't really a solution

Agreed, 150%. My comment had more to do with rejecting files than it did with embracing environ as a suitable alternative.

SQLite is likely the most trouble-free option at the moment.

With that being said, it would be nice to see Android's sys/system_properties.h ported to GNU/Linux proper and, from there, other Unixen.

> I would encourage you to play around with ghidra, just to see what return oriented programming and ARM limits does.

Having worked professionally in reverse engineering at DoD, I can assure you that this is something I'm intimately familiar with.

Files and environ are bad.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#358

Post by davidt84 »

kazinator wrote: post #354
davidt84 wrote: post #350

Earlier quoted context omitted.

But it is possible to safely use it in a single threaded program. There's no way to use it safely in a multi threaded application that may use setenv (unless you add your own synchronisation, and ensure everything uses it, even third party libraries).

Actually I don't believe that's the case. The getenv function as described by ISO C cannot be safely used in a program that only uses getenv, if that program uses ISO C threads, and more than one threat calls getenv without synchronizing with the others. I don't think POSIX fixes this: it doesn't specify that the environ array is protected against concurrent access. If two threads call getenv right around the same ti…

Hmm, I'm apparently correct for C++11, where calling getenv only is thread safe, but that's not guaranteed by earlier standards (or, as far as I can tell, by C or POSIX).

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#359

Post by kibwen »

loeg wrote: post #332
kibwen wrote: post #319

Earlier quoted context omitted.

And when can we expect the version of Posix that fixes setenv to be MT-safe?

You've shifted the goalposts from the comment I responded to.

Not at all, because if Posix wasn't set in stone, then surely it would take somewhat less than three-ish decades to fix this problem.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#360

Post by GoblinSlayer »

jrmg wrote: post #267
ChrisSD wrote: post #4

In the Rust std, `set_var` and `remove_var` will correctly require using an `unsafe {}` block in the next edition (2024). The documentation does now mention the safety issue but obviously it was a mistake to make these functions safe originally (albeit a mistake even higher level languages have made). https://doc.rust-lang.org/stable/std/env/fn.set_var.html There is a patch for glibc which makes `getenv` safe in more…

Wow, glibc now keep[s] older versions around and adopt[s] an exponential resizing policy. This results in an amortized constant space leak per active environment variable, but there already is such a leak for the variable itself (and that is even length-dependent, and includes no-longer used values). There have got to be pathalogical uses out there where this will cause unbounded memory growth in well-formed (accordi…

FWIW you can make a singly linked list with infinite number of nodes too. Memory leaks happen in well formed programs just fine, glibc is just one of many examples.
Post reply on HN