C stdlib isn't threadsafe and even safe Rust didn't save us • Hacker News Board

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#311

Post by hinkley » Thu, Jan 23, 2025, 6:44 PM UTC

bluGill wrote: post #279

dietr1ch wrote: post #172

Earlier quoted context omitted.

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

What is wrong with main setting those things first and then starting your main program? That is what everyone else does.

Not “everyone” does that. You have individual files doing their own initialization when they get loaded. Including loading other files or modules.

They might do it for testing purposes.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#312

Post by thayne » Thu, Jan 23, 2025, 6:54 PM UTC

usefulcat wrote: post #77

mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

In particular, it doesn't help if you call a c function that indirectly modifies the environment with FFI.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#313

Post by hinkley » Thu, Jan 23, 2025, 6:59 PM UTC

fch42 wrote: post #214

dietr1ch wrote: post #172

Earlier quoted context omitted.

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

You needn't go "hacky" for this; constructors for global/static variables are called before main(). But then, the underlaying linker support is usually "trivially exposed" (using the constructor attribute in gcc/clang, say). This (obviously?) isn't "110%" perfect as the order of the constructor calls for several such objects may not be well-defined, and were they to create threads (who am I to suggest being reasonabl…

JavaScript only just got top level async. So what I saw happen is that files that do their own background tasks start those either in their constructor or lazily in the case of static functions.

There was one place and only one place where we violated that, and it was in code I worked on. It was a low level module used everywhere else for bootstrapping, and so we collectively decided to do something sneaky in order to avoid making the entire code base async.

And while I find that most of the time people can handle making one special case for a rule, it was a complicated system and even “we” screwed it up occasionally for a good long while.

The problem was we needed to make a consul call at startup and the library didn’t have a synchronous way to make that call. So all bootstrapping code had to call a function and await it, before loading other things that used that module. At the end we had about a dozen entry points (services, dev and diagnostic tools). And I always got blamed because nobody seemed to remember we decided this together.

I hate singletons. And I ended up with one of only two in the whole project, and that hatred still wasn’t enough to prevent hitting the classical problems with singletons.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#314

Post by hinkley » Thu, Jan 23, 2025, 7:02 PM UTC

GoblinSlayer wrote: post #188

dietr1ch wrote: post #172

Earlier quoted context omitted.

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

grpc reads some configuration from environment; environment has portability problems too, so it's useful to set it to cross platform shape.

Some of the docker containers I made ended up having a bash shell as the entry point and I moved most of the environment variable init out of the code and into the script. But in dev sandbox some of that code runs without the script, so it was still a headache.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#315

Post by hinkley » Thu, Jan 23, 2025, 7:03 PM UTC

dietr1ch wrote: post #172

hinkley wrote: post #152

Earlier quoted context omitted.

I think there's a narrow window, at least in some programming languages, when environment variables can be set at the start of a process. But since it's global shared state, it needs to be write (0,1) and read many. No libraries should set them. No frameworks should set them, only application authors and it should be dead obvious to the entire team what the last responsible moment is to write an environment variable.…

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

Poor choice of phrasing.

I ended up implying some extra support when all I meant was “one could”.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#316

Post by msully4321 » Thu, Jan 23, 2025, 7:08 PM UTC

mad0 wrote: post #273

saagarjha wrote: post #210

Earlier quoted context omitted.

I am going to do this once, but not again. Please pay attention to it. You are not just wrong, but failing to demonstrate an understanding of the actual topic being discussed. I can't say whether you actually have it or not, but your responses do not demonstrate this. I have dealt with plenty of people on this site who say things that are factually incorrect, many of whom have argued with me when I do so. You are not…

Sounds like you are arguing with a bot? em-dashes are a giveaway (nobody sane uses these "—")

I go through periods of loving em dashes--but I always just write them as two dashes! (And LaTeX at least does the right thing.)

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#317

Post by mardifoufs » Thu, Jan 23, 2025, 7:14 PM UTC

nwellnhof wrote: post #245

> Our nightly CI machines run on Amazon AWS, which has the advantage of giving us a real, uncontainerized root user. > We don’t have the necessary files outside of the container, and our containers are quite minimal and don’t allow us to easily install gdb. Have people lost the ability to build and debug their code locally, without clouds and containers?

How would you debug locally when you probably don't have a device that runs the arch that is causing an issue? It's much faster to just debug in the actual environment where the failure happens anyways.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#318

Post by ChuckMcM » Thu, Jan 23, 2025, 7:23 PM UTC

harrall wrote: post #301

eqvinox wrote: post #96

Earlier quoted context omitted.

I have a different perspective: the underlying problem is calling setenv(). As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. It's not a mechanism for exchanging information within a process, as used here with SSL_CERT_FILE. And remember that the exec* family of calls has a version with an envp argument, which is what should be used if a child process is to be…

You can’t convince me that there is EVER a reason to call setenv() after program init as part of a regular program, outside needing to hack around something specific. Environmental variables are not a replacement for your config. It’s not a place to store your variables. Even if the env var API is fully concurrent, it is not convention to write code that expects an env var to change. There isn’t even a mechanism for…

As a really old school UNIX guy I'd agree with this. Programmatic manipulation of the environment is an 'attractive nuisance' in that I feel anything you might be trying to achieve by using the environment as a string scratch pad of things that are different for different threads, can be coded in a much safer way.

I'd be happy to have you copy the immutable read-only environment vector of strings into your space and then treat that as the source of such things.

I think it would be interesting to build all the packages with a stdlib that dumps core on any call to setenv() or unsetenv(). That would give one an idea of the scope of the problem.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#319

Post by kibwen » Thu, Jan 23, 2025, 7:25 PM UTC

loeg wrote: post #296

kibwen wrote: post #276

Earlier quoted context omitted.

The concern now is that, unlike when Posix was set in stone, threads exist.

The p in pthreads stands for Posix. I.e., uh, Posix is neither set in stone, nor entirely predates threads.

And when can we expect the version of Posix that fixes setenv to be MT-safe?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#320

Post by skissane » Thu, Jan 23, 2025, 8:02 PM UTC

account42 wrote: post #233

friendzis wrote: post #189

Earlier quoted context omitted.

`main` is the default entrypoint, with one simple argument to the linker you can change entrypoint symbol to whatever you wish. You can add `premain` function that calls `main` and set it as an entrypoint, you can implement pre-start logic in main and call main loop later. This is how any sane program is written anyway: set up environment -> continue with business logic

main() is not the entry pont, some platform specific CRT is.

Also, the CRT will call any functions you declare `__attribute__((constructor))` before it calls `main()`