Earlier quoted context omitted.
Actually I don't believe that's the case. The getenv function as described by ISO C cannot be safely used in a program that only uses getenv, if that program uses ISO C threads, and more than one threat calls getenv without synchronizing with the others. I don't think POSIX fixes this: it doesn't specify that the environ array is protected against concurrent access. If two threads call getenv right around the same ti…
Hmm, I'm apparently correct for C++11, where calling getenv only is thread safe, but that's not guaranteed by earlier standards (or, as far as I can tell, by C or POSIX).
C stdlib isn't threadsafe and even safe Rust didn't save us
361–370 of 370 posts
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#362Earlier quoted context omitted.
Specifically, any C or C++ library that calls setenv (despite documentation that says that setenv is not threadsafe).
Or any multithreaded program that uses a C or C++ library that calls setenv somewhere internally, and failed to document that it does so and is thus unsuitable for use by multithreaded programs. No library does that documentation, so you can't use libraries on POSIX systems if writing multithreaded code. Or you do and hope for the best. So everyone just hopes for the best.
Caveats: POSIX.1 does not require setenv() or unsetenv() to be reentrant.
...
Interface: setenv(), unsetenv()
Attribute: Thread safety
Value: MT-Unsafe const:env
Libraries that are thread-safe DO provide that documentation. One assumes that
libraries that don't provide that documentation are not thread=safe.GnuTLS docs: The GnuTLS library is thread safe by design, meaning that objects of the library such as TLS sessions, can be safely divided across threads as long as a single thread accesses a single object.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#363Earlier quoted context omitted.
The only thread-safe way to implement getenv/setenv as they currently exist is to leak the previous state when setenv allocates, such that existing pointers stay valid. The existing API simply lacks a mechanism to synchronize correctly. Leaking would be good enough for many use cases, but it would break long-running users of setenv (mainly those with libraries abusing env vars, as in TFA), and doesn't even solve how…
Do you mean pointers returned by getenv() ? Those could point to thread-local buffers that data gets copied into when getenv() is called.
If current platforms are safely making a copy of getenv before allowing their scheduler to interrupt, then yes I'd be ok with your solution.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#364Earlier quoted context omitted.
Hmm, I'm apparently correct for C++11, where calling getenv only is thread safe, but that's not guaranteed by earlier standards (or, as far as I can tell, by C or POSIX).
I'm surprised C++ would have anything to say about getenv ; mostly it just includes the standard C library via normative reference.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#365Earlier quoted context omitted.
Do you mean pointers returned by getenv() ? Those could point to thread-local buffers that data gets copied into when getenv() is called.
Only if we're willing to take for granted that a call to getenv invalidates the previous one. POSIX allows it, but I'm concerned about runtimes scheduling user tasks on the same thread. If current platforms are safely making a copy of getenv before allowing their scheduler to interrupt, then yes I'd be ok with your solution.
Worst case memory usage (all threads get all vars) is that you end up having a separate copy of the environment per thread, but it seems this is the best that can be done given the awful API.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#366Earlier quoted context omitted.
> environment related bug on linux, which is mysteriously less a problem on other unix's. How do you figure? The problem isn't the implementation, it's the API. setenv(), unsetenv(), putenv(), and especially environ, are inherently unsafe in a multithreaded program. Even getenv_r() can't really save you, since another thread may be calling setenv() while the (old) value of an env var is being copied into the provided…
>> environment related bug on linux, which is mysteriously less a problem on other unix's. > How do you figure? From https://illumos.org/man/3C/putenv : > The putenv() function can be safely called from multithreaded programs
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#367Earlier quoted context omitted.
I'm surprised C++ would have anything to say about getenv ; mostly it just includes the standard C library via normative reference.
Nevertheless, https://en.cppreference.com/w/cpp/utility/program/getenv
If anything calls the C getenv function, like a third party library, things are maybe not fine.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#368Earlier quoted context omitted.
Indeed, environment variables should be used to configure child processes, not to configure the current process, for non-shell programs, IMHO. Note that Java, and the JVM, doesn't allow changing environment variables. It was the right choice, even if painful at times.
I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.…
struct BeforeEnvFreeze(());
struct AfterEnvFreeze(());
impl BeforeEnvFreeze {
pub fn new() -> Self { /* singleton check using a static AtomicBool or something */ Self(()) }
pub fn freeze(self) -> AfterEnvFreeze { AfterEnvFreeze(()) }
pub fn set_env(&self, ...) { ... }
}
impl AfterEnvFreeze {
pub fn spawn_thread(&self, ...) { ... }
}
fn main() {
let a = BeforeEnvFreeze::new();
a.set_env(...);
a.set_env(...);
//b.spawn_thread(...); // not available
let b = a.freeze(); // consumes `a`
b.spawn_thread(...);
//a.set_env(...); // not available
}
Exercises left to the reader:• Banning access to the relevant bits of Rust's stdlib, libc, etc. as a means of escaping this "safe" abstraction
• Conning your lead developer into accepting your handwave
• Setting up the appropriate VCS alerts so you have a chance to NAK "helpful" "utility" pull requests that undermine your "protections"
And of course, this all remains a hackaround for POSIX design flaws - your engineering time might be better spent ensuring or enforcing your libc is "fixed" via intentional memory leaks per e.g. https://github.com/bminor/glibc/commit/7a61e7f557a97ab597d6f... , which may ≈fix more than your Rust programs.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#369Earlier quoted context omitted.
Environment variables are a gigantic, decades-old hack that nobody should be using... but instead everyone has rejected file-based configuration management and everyone is abusing environment variables to inject config into "immutable" docker containers...
> instead everyone has rejected file-based configuration management With good reason. Files are surprisingly hard: https://danluu.com/deconstruct-files/
For configuration files, the write-fsync-move strategy works fine. Generally you don't need fsync, since most people don't use the file system settings that allow data writes to be reordered with the metadata rename.
Re: C stdlib isn't threadsafe and even safe Rust didn't save us
#370Earlier quoted context omitted.
It's threadsafe in the memory sense. It's not threadsafe in the having an idea what files you are accessing sense.
The latter is always true even when you don't use chdir(2) and/or always use absolute file paths since, you know, there are other processes that can re-arrange the file system whatsoever way the like. The file system is one example of the unavoidable global mutable shared state (another example is network) which one simply has to deal with.
If your sensitive logs end up in the webserver root because one thread used chdir to temporarily change the working directory it's on the application writer.
Or to put it another way, the filesystem as a whole being shared mutable state does not make the current working directory being shared mutable state between threads any less of an issue.