Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C stdlib isn't threadsafe and even safe Rust didn't save us

edgedb.com

Post reply on HN
Print view View original

251–260 of 370 posts

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#251

Post by bluGill »

nwellnhof wrote: post #245

> Our nightly CI machines run on Amazon AWS, which has the advantage of giving us a real, uncontainerized root user. > We don’t have the necessary files outside of the container, and our containers are quite minimal and don’t allow us to easily install gdb. Have people lost the ability to build and debug their code locally, without clouds and containers?

This is a random trash only on arm. I doubt they could get the crash to happen locally - most likely their developer machines were all x86 where it never crashed.

they should have handled crashes better - a problem they seem to recognize but not the issue here so not covered.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#252

Post by bluGill »

api wrote: post #248
nwellnhof wrote: post #245

> Our nightly CI machines run on Amazon AWS, which has the advantage of giving us a real, uncontainerized root user. > We don’t have the necessary files outside of the container, and our containers are quite minimal and don’t allow us to easily install gdb. Have people lost the ability to build and debug their code locally, without clouds and containers?

Yes. It’s shocking just how much cloud SaaS has distorted peoples understanding of things. You need all kinds of layers of cloud complexity and deployment to do the most trivial stuff. We have 100% reversed the PC revolution and returned to the era of clunky expensive mainframe computing. The reason is that cloud is where all the money is because cloud is DRM. Put software there and you can charge a subscription and…

There is a lot to like about the clould model as a user. I can access my data where ever I am, from what ever device I have, and I won't lose it to a disc crash.

there are faults to the cloud but it solves real problems users have.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#253

Post by gpderetta »

fch42 wrote: post #218
GoblinSlayer wrote: post #188

Earlier quoted context omitted.

grpc reads some configuration from environment; environment has portability problems too, so it's useful to set it to cross platform shape.

The "cross platform" way of setting the environment is to set it "from outside" of the program - meaning, through the executor, whether that's the shell or the container runtime or even the kernel commandline if you insist to rewrite init in rust/go/zig/... It can be as-easy-as spawning your process via "env -i VAR1=... ... myprogram ..." - and given this also clears the dangers of env-insertion exploits, it's good p…

At the limit a program can execve itself with the new env.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#254

Post by gpderetta »

o11c wrote: post #175
tsimionescu wrote: post #167

Earlier quoted context omitted.

This is actually not that hard to fix. Getenv() could keep several copies of the value around: one internal copy protected by a mutex, that it never returns, and one copy per thread that it stores in thread local storage. When you call getenv(), it locks the mutex, checks if the current thread's value exists, populates it from the internal copy if not, and returns it. It will also install a new setenv-specific signal…

If you only consider `getenv`/`setenv` there are indeed many solutions, but it's not that simple. You also need to consider `putenv` (not that nasty, you just need to treat it like initial environment, which means you can't use a single range check) and accessing the `environ` variable directly (nasty). Your particular solution doesn't work because people expect `getenv` to be async-signal-safe, which means you shoul…

> accessing the `environ` variable directly (nasty).

"easy": protect the page containing environ and handle the mutation from the signal handler.

/s of course.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#255

Post by pornel »

aragilar wrote: post #216
vlovich123 wrote: post #148

Earlier quoted context omitted.

> aka don't let the perfect be the enemy of the good Exactly my point. Over time *environ would disappear, at least from the major software projects that everyone uses (assuming it's even in use in them in the first place).

That still doesn't mean getenv would be safe. Unless you know nothing uses **environ (e.g. by breaking the ABI, which no-one will do because it'll break everything), you can't rely on getenv being safe.

There should be locking getters/setters for the environ, and all users should switch to them.

Yes, it will take a long time, and some users will complain it doesn't work on their PDP-11, but the problem will never be solved if there's no migration path to a safe solution.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#256

Post by josefx »

fch42 wrote: post #229
josefx wrote: post #143

Earlier quoted context omitted.

C could provide functions to lock/unlock a mutex and require that any attempt to access the environment has to be done holding the mutex. This would still leave the correctness in the hands of the user, but at least it would provide a standard API to secure the environment in a multi threaded application that library and application developers could adopt.

That is basically "what it means" if an interface is non-MT: you can call this no-problem if you know you're singlethreaded, and if you're not, find your own way to serialize (meaning: have your own locking prinitive you acquire/release where you make calls to these functions). One could "dream of" a func that tells libc "acquire/drop this mutex of mine around get/set/putenv calls" but that'd simply move the problem…

> because the nifty "frameworks"

Malicious software exists, does that mean we should remove all threading primitives from the standard?

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#257

Post by OskarS »

tsimionescu wrote: post #167
debugnik wrote: post #132

Earlier quoted context omitted.

No amount of locking can make the getenv API thread-safe, because it returns a pointer which gets invalidated by setenv, but lacks a way to release ownership over it and unblock setenv safely (or to free a returned copy). So setenv's existence makes getenv inherently unsafe unless you can ensure the entire application is at a safe point to use them.

This is actually not that hard to fix. Getenv() could keep several copies of the value around: one internal copy protected by a mutex, that it never returns, and one copy per thread that it stores in thread local storage. When you call getenv(), it locks the mutex, checks if the current thread's value exists, populates it from the internal copy if not, and returns it. It will also install a new setenv-specific signal…

> it should also offer a new API that simply returns a copy every time

Returning a copy isn't great (memory allocation!), the API should probably be something like:

    int getenv(const char *varName, char *buf, size_t bufSize, size_t *varSize);
Where the caller manages the buffer and getenv writes into it (so it can e.g. be stack or statically allocated), the third argument is the size of the caller-managed buffer, then the last variable is an "out parameter" that returns the "true" length of the environment variable. Then afterwards, you can check if `*varSize > bufSize`, and if so, you need to make your buffer larger. The return value is an error code.

Doing it like this, you can easily implement the "return a malloced copy" if you want to, but it also gives you the option to avoid allocation entirely. This is important for e.g. embedded or real-time applications, or anything that just likes to avoid `malloc()/free()`.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#258

Post by api »

bluGill wrote: post #252
api wrote: post #248

Earlier quoted context omitted.

Yes. It’s shocking just how much cloud SaaS has distorted peoples understanding of things. You need all kinds of layers of cloud complexity and deployment to do the most trivial stuff. We have 100% reversed the PC revolution and returned to the era of clunky expensive mainframe computing. The reason is that cloud is where all the money is because cloud is DRM. Put software there and you can charge a subscription and…

There is a lot to like about the clould model as a user. I can access my data where ever I am, from what ever device I have, and I won't lose it to a disc crash. there are faults to the cloud but it solves real problems users have.

There are other ways that could be achieved, like cloud storage constantly mirroring local but encrypted with local keys or keys controlled by the user.

This is the iCloud model and it works. Imagine a more open version with competing storage providers.

This, however, would hand control back to the user, which would be bad for the software industry with its addiction to lock in and recurring revenue.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#259

Post by fch42 »

josefx wrote: post #256
fch42 wrote: post #229

Earlier quoted context omitted.

That is basically "what it means" if an interface is non-MT: you can call this no-problem if you know you're singlethreaded, and if you're not, find your own way to serialize (meaning: have your own locking prinitive you acquire/release where you make calls to these functions). One could "dream of" a func that tells libc "acquire/drop this mutex of mine around get/set/putenv calls" but that'd simply move the problem…

> because the nifty "frameworks" Malicious software exists, does that mean we should remove all threading primitives from the standard?

Obviously not, but _threading_ primitives are not the subject of this post at all. Declared-as Non-threadsafe interfaces are. And of course one (as is happening here) one can argue whether all "system runtimes" shall be threadsafe. Right now though, they are not, and agreed/sanctioned standards don't require them to be. Again (also as happening here) opinions may differ whether changes-to-make-threadsafe would be bugfixes, enhancements, or (require) new interfaces. I have expressed my views on this. Happy to agree to disagree, though.

Re: C stdlib isn't threadsafe and even safe Rust didn't save us

#260

Post by fch42 »

gpderetta wrote: post #253
fch42 wrote: post #218

Earlier quoted context omitted.

The "cross platform" way of setting the environment is to set it "from outside" of the program - meaning, through the executor, whether that's the shell or the container runtime or even the kernel commandline if you insist to rewrite init in rust/go/zig/... It can be as-easy-as spawning your process via "env -i VAR1=... ... myprogram ..." - and given this also clears the dangers of env-insertion exploits, it's good p…

At the limit a program can execve itself with the new env.

Indeed, and from my point of view, that's perfectly ok.
Post reply on HN