Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

231–240 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#231

Post by dvtkrlbs »

IshKebab wrote: post #57
zanderwohl wrote: post #26

Earlier quoted context omitted.

It would be a tremendous amount of work, and would take years. Meanwhile, the problems are avoidable. It's not exactly the "rust way" to just remember and avoid problems, but everything in language design is compromises.

"Impossibru!!" https://github.com/sunfishcode/eyra Oh look: > Why use Eyra? It fixes Rust's set_var unsoundness issue. The environment-variable implementation leaks memory internally (it is optional, but enabled by default), so setenv etc. are thread-safe.

If I understand it correctly this still doesnt help with downstream dependencies.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#232

Post by belter »

usefulcat wrote: post #77
mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

> It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

A mutex can ensure thread safety but risks deadlocks if not used carefully and will hurt performance...

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#233

Post by account42 »

friendzis wrote: post #189
dietr1ch wrote: post #172

Earlier quoted context omitted.

Which programming languages? When using C++ I wanted programs to have a function that was called before main() and set up things that got sealed afterwards, like parsing command-line-arguments, the environment variables, loading runtime libraries, and maybe look at the local directory, but I'm not sure if it'll be a useful and meaningful distinction unless you restructure way too many things. I remember that on the F…

`main` is the default entrypoint, with one simple argument to the linker you can change entrypoint symbol to whatever you wish. You can add `premain` function that calls `main` and set it as an entrypoint, you can implement pre-start logic in main and call main loop later. This is how any sane program is written anyway: set up environment -> continue with business logic

main() is not the entry pont, some platform specific CRT is.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#234

Post by fch42 »

miohtama wrote: post #86
masklinn wrote: post #21

Earlier quoted context omitted.

Because the std implementation can not force synchronisation on the libc, so any call into a C library which uses getenv will break... which is exactly what happened in TFA: `openssl-probe` called env::set_var on the Rust side, and the Python interpreter called getenv(3) directly.

Is it possible to skip libc completely or would this introduce too many portability concerns?

In short, no - because environment variables are userland state only, you can't interact with them using system calls, the kernel doesn't keep a "canonical" copy of them on behalf of the process. So the "environment" is part of libc, and "libc's way" of interacting with it at runtime "is the way".

From the syscall interface point of view ... you pass the initial env of a process when you exec(), and the kernel copies that to (userland) memory of the new process. The fact "default initialisation" can copy from the environment of the exec()'ing parent, or the fact that the kernel can "read" a process' env (see /proc//environ) doesn't change this; the kernel needn't be "accommodating" all the possible and impossible ways how a user application may want to interact with that state there, if you mess-too-much with it, you get garbage. Sooo ... the portability wart is setenv(), because as far as the system is concerned... your "initial" env is passed to you when exec() is called, and any modification thereafter is your concern, your problem, but foremost, your choice. And choices come with taking responsibility for the ones you make.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#235

Post by hamandcheese »

belter wrote: post #232
usefulcat wrote: post #77

Earlier quoted context omitted.

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

> It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex. A mutex can ensure thread safety but risks deadlocks if not used carefully and will hurt performance...

Agree about performance, but wouldn't there need to be >1 mutex to risk a deadlock?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#237

Post by account42 »

alkonaut wrote: post #208
tsimionescu wrote: post #167

Earlier quoted context omitted.

This is actually not that hard to fix. Getenv() could keep several copies of the value around: one internal copy protected by a mutex, that it never returns, and one copy per thread that it stores in thread local storage. When you call getenv(), it locks the mutex, checks if the current thread's value exists, populates it from the internal copy if not, and returns it. It will also install a new setenv-specific signal…

There has to be some sort of nuance regarding why this seemingly simple fix hasn't been made yet. Changing from crashing to blocking doesn't seem like a big breaking change.

Because it doesn't actually solve anything: You're still replacing whatever getenv returned from under the nose the program code - if that happens in another thread or in a signal handler in the same thread doesn't make any difference.

And that's before you even get to the `extern char *environ` global.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#238

Post by nemetroid »

mmastrac wrote: post #46
Animats wrote: post #43

Earlier quoted context omitted.

Yes. That's known. Most of the rest of the problem here seems to be the development environment. They're testing on a remote machine in an Amazon data center and using Docker. This rig fails to report that a process has crashed. Then they don't have enough debug symbol info inside their container to get a backtrace. If they'd gotten a clean backtrace reported on the first failure, this would have been obvious. Why is…

Yup, it's mostly just the story and tools we used to get ourselves out of a mess that was made harder by some decisions made earlier -- the tests were running in a container with stripped symbols (we're going to ship symbols after this, no reason to over-optimize), our custom test runner failed to report process death (an oversight). There's no reason setenv should have been called here. The `openssl-probe` library c…

> we're going to ship symbols after this, no reason to over-optimize

You might want to look into debuginfod.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#239

Post by account42 »

Joker_vD wrote: post #105
pshc wrote: post #92

Earlier quoted context omitted.

The underlying problem is that setenv is mutable global state and should never have existed

The process's current directory is mutable global state as well, and yet chdir(2) is thread-safe.

It's threadsafe in the memory sense. It's not threadsafe in the having an idea what files you are accessing sense.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#240

Post by xxs »

alkonaut wrote: post #208
tsimionescu wrote: post #167

Earlier quoted context omitted.

This is actually not that hard to fix. Getenv() could keep several copies of the value around: one internal copy protected by a mutex, that it never returns, and one copy per thread that it stores in thread local storage. When you call getenv(), it locks the mutex, checks if the current thread's value exists, populates it from the internal copy if not, and returns it. It will also install a new setenv-specific signal…

There has to be some sort of nuance regarding why this seemingly simple fix hasn't been made yet. Changing from crashing to blocking doesn't seem like a big breaking change.

B/c you never need setenv outside a single threaded command line utilities, and even then it's questionable.
Post reply on HN