Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

341–350 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#341

Post by kazinator »

Joker_vD wrote: post #105
pshc wrote: post #92

Earlier quoted context omitted.

The underlying problem is that setenv is mutable global state and should never have existed

The process's current directory is mutable global state as well, and yet chdir(2) is thread-safe.

chdir is only thread safe to the extent that corruption won't occur.

If one thread is using relative paths, and another is doing a chdir-based traversal (as using the nftw function, for instance), that first thread's accesses are messed up.

This is why POSIX now has various -at functions; the provide stable relative access.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#342

Post by kazinator »

davidt84 wrote: post #109
fsckboy wrote: post #45

Earlier quoted context omitted.

you've gotten a lot of answers which say the same thing, but which I don't think answer your question: synchronization methods impose various complexity and performance penalties, and single threaded applications which don't need that would pay those penalties and get no benefit. Unix was designed around a lightweight ethos that allowed simple combining of functions by the user on the command line. See "worse is bett…

The real problem is that getenv() and setenv() were created before threads were really a thing.

getenv can easily be misused in a single threaded program.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#343

Post by jamesfinlayson »

xxs wrote: post #241
jamesfinlayson wrote: post #144

Earlier quoted context omitted.

Sure is painful (mostly when writing tests where the environment variables aren't abstracted in some way). But I think it was actually possible to hack around up until Java 17.

if you really wish - you can change the bootstrap path and allow changing env() for whatever reason you want to (likely via copy on write). If you don't wish to do that feel free to spawn a child process with whatever env you desire, then redirect/join sys in/our/err (0/1/2) Those are trivial things in around 100 lines of code and have been available since System.getenv() got back (it used to be deprecated and non-fu…

A lot of the Java I'm writing is in AWS Lambda so my options are a bit more limited.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#344

Post by skissane »

eqvinox wrote: post #114
Joker_vD wrote: post #110

Earlier quoted context omitted.

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv. Mutating argv is actually quite popular, or at least it used to be.

Yes, and if there were "setargv()" or "getargv()" functions, they'd have the same issues ;) … but argv is a function parameter to main()¹, and only that. ¹ or technically whatever your ELF entry point is, _start in crt0 or your poison of choice.

On Linux, a privileged process can change the memory address which the kernel (/proc filesystem) reads argv/etc from... prctl(PR_SET_MM) with the PR_SET_MM_ARG_START/PR_SET_MM_ARG_END arguments. Likewise, with PR_SET_MM_ENV_START/PR_SET_MM_ENV_END.

The API is ugly, and since it needs CAP_SYS_RESOURCE many programs can't use it... but systemd does: https://github.com/systemd/systemd/blob/2635b5dc4a96157c2575...

This shouldn't cause the kind of race conditions we are talking about here, since it isn't changing a single arg, it is changing the whole argv all at once. However, the fact that PR_SET_MM_ARG_START/PR_SET_MM_ARG_END are two separate prctl syscalls potentially introduces a different race condition. If Linux would only provide a prctl to set both at once, that would fix that. The reason it was done this way, is the API was originally designed for checkpoint-restore, in which case the process will be effectively suspended while these calls are made.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#345

Post by liontwist »

kazinator wrote: post #340
liontwist wrote: post #242

Earlier quoted context omitted.

Please no. If your program wants to use the environment as an out-of-band global var for cross thread communication, you can make your own mutex.

That will break if any code which is not aware of the mutex calls getenv , even for a variable not related to the communication.

Of course. It’s a bad idea. If you modify it the environemnt becomes a global var.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#346

Post by skissane »

adrian_b wrote: post #131
tedunangst wrote: post #58

Earlier quoted context omitted.

Who has a signal safe malloc?

POSIX does not require malloc to be signal safe. Therefore I do not think that anyone has bothered to implement a signal-safe malloc, as this is likely to be complicated. Allocating memory in a signal handler makes no sense in a well designed program, so not being allowed to use malloc and related functions is not a problem.

On Linux, mmap() is your signal-safe malloc.

POSIX doesn't require mmap() to be async-signal safe, but on Linux it de facto is.

Of course, doing every memory allocation at the kernel level is going to be very slow and resource intensive. But if it isn't on a hot code path...

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#347

Post by pjmlp »

loeg wrote: post #331
pjmlp wrote: post #326

Earlier quoted context omitted.

I am old enough to remeber when UNIX only had processes, and several thread designs were being discussed until eventually pthreads one design won. POSIX predates adoption of threads in the UNIX world.

Yes, yes, but also post-dates adoption of threads. It's an ongoing standard.

Which only standardises existing practice, not inventing new ideas.

So first everyone implementing POSIX has to agree to move forward with thread safety where it fails short.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#349

Post by api »

bluGill wrote: post #335
api wrote: post #329

Earlier quoted context omitted.

For CPU power, a Raspberry Pi today is faster than servers that ran whole medium to large businesses 20 years ago. Much of what people do with SaaS involves backend processes that could run on a 1990s era PC. There are exceptions, like large AI models and huge databases like web search, though in the case of AI models I can run pretty decent ones locally already, but on an admittedly expensive laptop. If the rate at…

Are you willing for doom quality? It got the job done, but compared to a modern 3d game it looks really bad. Of course the elephant in the room remains: I need my data where I am.

Modern 3D games are local, so I’m not sure the point there. My point about 90s machines was that most business SaaS is not compute or data heavy unless it’s for a huge corporation.

As far as local data: my laptop has terabytes, my phone over a hundred gigabytes. I have fiber at home and have seen speeds approaching a gigabit on 5G.

It’s not that often that people sit down at entirely unfamiliar machines they’ve never used, log in, and try to do data intensive work. In that case I suppose an iCloud model of compute would be downloading a lot.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#350

Post by davidt84 »

kazinator wrote: post #342
davidt84 wrote: post #109

Earlier quoted context omitted.

The real problem is that getenv() and setenv() were created before threads were really a thing.

getenv can easily be misused in a single threaded program.

But it is possible to safely use it in a single threaded program.

There's no way to use it safely in a multi threaded application that may use setenv (unless you add your own synchronisation, and ensure everything uses it, even third party libraries).

Post reply on HN