C stdlib isn't threadsafe and even safe Rust didn't save us • Hacker News Board

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#241

Post by xxs » Thu, Jan 23, 2025, 11:58 AM UTC

jamesfinlayson wrote: post #144

diroussel wrote: post #107

Earlier quoted context omitted.

Indeed, environment variables should be used to configure child processes, not to configure the current process, for non-shell programs, IMHO. Note that Java, and the JVM, doesn't allow changing environment variables. It was the right choice, even if painful at times.

Sure is painful (mostly when writing tests where the environment variables aren't abstracted in some way). But I think it was actually possible to hack around up until Java 17.

if you really wish - you can change the bootstrap path and allow changing env() for whatever reason you want to (likely via copy on write). If you don't wish to do that feel free to spawn a child process with whatever env you desire, then redirect/join sys in/our/err (0/1/2)

Those are trivial things in around 100 lines of code and have been available since System.getenv() got back (it used to be deprecated and non-functional prior Java 1.5 or 2004)

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#242

Post by liontwist » Thu, Jan 23, 2025, 12:02 PM UTC

usefulcat wrote: post #77

mmastrac wrote: post #5

The major takeaway from this is that Rust will be making environment setters unsafe in the next edition. With luck, this will filter down into crates that trigger these crashes ( https://github.com/alexcrichton/openssl-probe/issues/30 filed upstream in the meantime).

But that won't actually fix the underlying problem, namely that getenv and setenv (or unsetenv, probably) cannot safely be called from different threads. It seems like the only reliable way to fix this is to change these functions so that they exclusively acquire a mutex.

Please no.

If your program wants to use the environment as an out-of-band global var for cross thread communication, you can make your own mutex.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#243

Post by xxs » Thu, Jan 23, 2025, 12:05 PM UTC

kbolino wrote: post #62

shikon7 wrote: post #24

I wonder why it is so hard for Rust to implement its own safe stdlib independent of C.

They did, it's called core. But it assumes no operating system at all, and environment variables require an operating system.

>and environment variables require an operating system.

Yes to read them, if Rust wish to modify - modify your own, already copied structure. I'd do that in pretty much any language.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#244

Post by liontwist » Thu, Jan 23, 2025, 12:08 PM UTC

josefx wrote: post #128

pshc wrote: post #92

Earlier quoted context omitted.

The underlying problem is that setenv is mutable global state and should never have existed

Welcome to the C standard library, the application of mutable global state to literally everything in it has to be the most consistent and predictable feature of the language standard.

I used to think this was bad too. But when C was designed an entire single threaded program was considered the unit of encapsulation for functionality. Now it’s mostly libraries.

The former allows you to design a coherent system. a lot of design questions which are annoying (“how do I access config data consistently, etc) become very clear.

It also makes C more productive. If global vars and static locals are unbanned, features like closures become less important.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#245

Post by nwellnhof » Thu, Jan 23, 2025, 12:10 PM UTC

> Our nightly CI machines run on Amazon AWS, which has the advantage of giving us a real, uncontainerized root user.

> We don’t have the necessary files outside of the container, and our containers are quite minimal and don’t allow us to easily install gdb.

Have people lost the ability to build and debug their code locally, without clouds and containers?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#246

Post by xxs » Thu, Jan 23, 2025, 12:12 PM UTC

lmm wrote: post #180

PaulDavisThe1st wrote: post #90

Earlier quoted context omitted.

One of the major differences between X Window and the win32 GUI APIs is that the windows one builds in thread safety, and it cannot be removed. This means that you pay the price of mutexes and the like (what the windows world likes to call "critical sections"), even if you have a single threaded GUI. X Window, on the other hand, decided to do nothing about threads at all, leaving it up to the application. 30 years af…

> Consequently, the overhead present in the win32 API is basically just dead-weight, there to make sure that "things are safe by default". How much overhead is it though? IIRC uncontended mutexes are practically free, especially when they're only being used from a single thread. Our industry is way too eager to make things unsafe for the sake of marginal performance differences that are irrelevant for most use cases,…

uncontended mutexes are very cheap but not free. lock cmpxchg has way higher latency (and coherency traffic) costs than a simple move or (xchg). Java had lock elision, effectively trying to solve the hardware problem in software back in mid 00s. There are optimizations to be made it running on a single core (no need for the lock), e.g. docker with taskset

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#247

Post by gobblegobble2 » Thu, Jan 23, 2025, 12:15 PM UTC

GoblinSlayer wrote: post #206

saagarjha wrote: post #190

Earlier quoted context omitted.

A big reason to mutate argv is to change the process's name for tools like top.

For that you write to /proc/self/comm, that's where top gets it from.

It may work for top, but not ps among others. The only reliable way is clobbering argv. That's just the way it is. In my opinion, glibc should finally provide setproctitle(), so programs like postgresql or chrome (https://source.chromium.org/chromium/chromium/src/+/main:bas...) don't have to resort to argv hacks.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#248

Post by api » Thu, Jan 23, 2025, 12:17 PM UTC

nwellnhof wrote: post #245

> Our nightly CI machines run on Amazon AWS, which has the advantage of giving us a real, uncontainerized root user. > We don’t have the necessary files outside of the container, and our containers are quite minimal and don’t allow us to easily install gdb. Have people lost the ability to build and debug their code locally, without clouds and containers?

Yes. It’s shocking just how much cloud SaaS has distorted peoples understanding of things. You need all kinds of layers of cloud complexity and deployment to do the most trivial stuff. We have 100% reversed the PC revolution and returned to the era of clunky expensive mainframe computing.

The reason is that cloud is where all the money is because cloud is DRM. Put software there and you can charge a subscription and nobody can evade it and you have perfect lock in forever. People usually can’t even get their data out. You can also do all kinds of realtime analytics conveniently to optimize your product.

Computing architecture is downstream of the business model. Mainframe died originally because there was no Internet and PCs were cheaper, but vendors also lost a lot of their lock in power. Now they have a way to bring a model that is much more profitable back. No more pesky freedom for users, who to be fair if given such freedom will often just refuse to pay, making quality software a non-viable business.

Tangent I know.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#249

Post by xxs » Thu, Jan 23, 2025, 12:17 PM UTC

tsimionescu wrote: post #163

kelnos wrote: post #80

Earlier quoted context omitted.

That doesn't solve anything. You could be using a library (perhaps a closed-source one) that doesn't use these hypothetical lockenv()/unlockenv() functions. This needs to be fixed inside libc, but there's no way to do so completely without breaking backward-compatibility.

Yes, I was talking about fixes inside libc. The poster above was claiming it can't be done inside libc. And the lockvenv/unlockvenv functions I was mentioning were meant to exist besides the internal locking inside setenv/getenv. They would only be used if you needed transactional access (a combination of setting/getting multiple env vars atomically).

using copy on write would be easier (and more performant), along with getenv_r. POSIX requires not to copy the data which makes the entire mutex/lock or CoW pointless. Of course, there will be the mandatory mentioning of "extern char *environ;"[0]. That returns are raw C strings as you can find them.

What could work is per thread env. changes - but that's not likely to happen

[0]: https://linux.die.net/man/3/environ

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#250

Post by Joker_vD » Thu, Jan 23, 2025, 12:25 PM UTC

fch42 wrote: post #226

eqvinox wrote: post #119

Earlier quoted context omitted.

I mean, yes, we're in "violent agreement" there. It's nice that libc squirrels away a copy and gives you a `getenv()` function with a string lookup, but… setenv… that was just a horrible idea. It's not really wrong to view it as a tool that allows you to muck around with main()'s local variables. Which to me sounds like one should take a shower after using it ;D (Ed.: the man page should say "you are required to take…

+1 on the "horrible idea" part. Thing is, the (history of the) UNIX APIs - call'em "libc" if you like - is littered with the undead corpses of horrible ideas. Who thought that having global file write offsets are great ? Append-only writes ? Global working directories ? The ability to write the password db via putpwent() ? Modifying your own envp or argv ? Why have a horribly-scaling hack like fcntl-based file lockin…

Append-only writes are actually amazing, having several processes writing into the same file and have their writes interleaved instead of destroying each other is almost impossible to re-create in the user space.

And I still don't understand why processes "modifying their own envp or argv" are met with such revulsion in this comment thread except from the "I dislike that on ideological grounds" reason. Now, the ability to modify envp and/or argv of other processes while those are running, yes, that's a horrible idea. But modifying your own internal process state?

Oh, and fcntl file locks are horrible for the historical reasons: basically, when POSIX (or its predecessor?) were trying to decide on a portable interface, the representative of one of the vendors cobbled together this API and its implementation in a week or two, and then showed to the meeting with it. To his surprise, instead of arguing everyone else basically said "eh, looks fine", and that was it, we now have broken "why on earth does close()/fork()/exec() interact with locks like that" behaviour.