Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Problem with Friendly C

blog.regehr.org

Post reply on HN
Print view View original

71–80 of 174 posts

Re: The Problem with Friendly C

#71

Post by superuser2 »

tamana wrote: post #54
scott_s wrote: post #33

Earlier quoted context omitted.

The problem, as Regehr (the author of this submission) has pointed out many times before, is more subtle than that. Because certain behavior is considered undefined, compilers are allowed to assume the code it is compiling is well defined, and optimize accordingly. That can cause simple bugs to have mysterious effects. For example: a->thing = 42; if (a == NULL) { return; } Obviously that code is wrong; I should check…

But the compiler could just as well emit a warning "useless NULL comparison" instead of blithely assuming that the programmer intentionally wrote a book expression. That wouldn't handle every case of UB, but would handle many.

A warning on removed dead code isn't helpful because dead come is legitimately removed all the time. No one would ever heed it.

Re: The Problem with Friendly C

#72

Post by derefr »

nly wrote: post #62
derefr wrote: post #44

Earlier quoted context omitted.

How about being build-system defined? As in, let me specify a "target behavior profile" as a switch to the compiler, which would be shorthand for a bunch of switches that define specific results for specific undefined behaviors: "-foverflow=checked" (add explicit checks) vs. "-foverflow=wrap" (do what most ISAs already do, but even on target ISAs that don't do that) vs. "-foverflow=clip" (do something weird that no t…

GCC has an intermediate representation. It's just not exercised as a separate project (like LLVM). It's far from "single pass"

You're right; to be clear, what I was talking about above was whether the passes are decoupled at an abstract level, not so much whether the implementation uses multiple phases.

You can take LLVM "bitcode" as the product of clang, move it to another system with a different arch, and finish compiling the IR to a native binary on that system. This implies that the first step and second step don't share any state other than the IR itself—meaning that flags passed directly to clang in the first phase can't affect codegen, other than by influencing the generated IR; and that codegen has to treat the IR (and the environment) as its sole inputs.

Whereas with GCC, as far as I know, the RTL IR isn't arch-neutral, and the GENERIC/GIMPLE IRs aren't fully "baked" yet (i.e. they're compiler->compiler step IRs, not post-compilation pre-codegen IRs.) So there's no conceptual guarantee (even if there's an implementation-level one) that the compiler won't use compile-phase state to inform codegen.

In other words: it's guaranteed that you'd be able to implement the switches I'm talking about in clang+LLVM. You might (but won't necessarily) have to do some work to get GCC into the right shape to implement them.

Re: The Problem with Friendly C

#73

Post by caf »

nkurz wrote: post #51
caf wrote: post #12

Earlier quoted context omitted.

You have to look deeper for the OOB array access question. For example, imagine I have code like this: if (a > 1) b++; array[b] = 0; c = a + 10; Under your suggested semantics, can the compiler use the value loaded from `a` at the point of the if() statement to calculate `a + 10`? Or does it have to emit a reload of `a` after the array access, in case `array[b]` was an OOB access that overwrote `a`?

I'm not seeing the deeper question, whereas what 'userbinator' said still makes sense to me. The compiler's implementation defined behavior would simply be "if you tell me to write a value to an address, I'll create assembly that tries to write that value to that address". That's all. If this address is outside the allocated range of the array, there is no guarantee that it's safe that write this value, or that it wo…

When you're saying that the compiler should be allowed to optimise under the assumption that OOB array accesses don't clobber other variables, this means that OOB array accesses can make very weird things happen indeed. For example:

  if (a > 1)
      b++;
  array[b] = 0;
  if (a > 1)
      foo(a);
We might see foo() called with argument 0 when that's apparently impossible - the OOB write has apparently "reached forward" (it can also "reach backward").

This is why it ends up just being "undefined behaviour" - to do better either you have to somehow exhaustively document all the kinds of weird things that can happen ("it writes to the memory" isn't enough, because of the way that can interact with the optimiser) or you have to unreasonably constrain the optimiser.

Re: The Problem with Friendly C

#74

Post by caf »

Dylan16807 wrote: post #35
caf wrote: post #12

Earlier quoted context omitted.

You have to look deeper for the OOB array access question. For example, imagine I have code like this: if (a > 1) b++; array[b] = 0; c = a + 10; Under your suggested semantics, can the compiler use the value loaded from `a` at the point of the if() statement to calculate `a + 10`? Or does it have to emit a reload of `a` after the array access, in case `array[b]` was an OOB access that overwrote `a`?

How about treating it just like another thread accessing the values? So yes in the general case, no if you put in memory barriers or atomics.

Race conditions are generally treated as "you get undefined behavior" for much the same reasons.

Re: The Problem with Friendly C

#75

Post by Animats »

xamuel wrote: post #68

You have to choose whether or not you want the compiler to generate code that spends time doing things that the programmer didn't ask for. If you write a library with a function which accepts variables x and y and computes x[y] (or x must add some kind of branching logic in there to check what's passed in at runtime. In other words, spend time doing things the programmer didn't ask for. Maybe when we make more progre…

"Maybe when we make more progress with formal proof-generating languages, we can create a "friendly" C where the compiler refuses to compile the code until it's accompanied by formal proofs of UB-avoidance."

That's entirely feasible. I headed a team which did that for a dialect of Pascal over three decades ago.[1] It's since been done for Modula III, Java, Spec#, and Microsoft Windows drivers.

One reasonable thing to do is to have the compiler generate run-time assertions for every statement which has a precondition for defined behavior. All pointer dereferences get "assert(p != 0)". The shift problem in the original article gets "assert(n > 0); assert(n Then you try to optimize out the run-time checks, or at least hoist them out of loops. A simple prover can remove about 90% of the checks without much effort.

[1] http://www.animats.com/papers/verifier/verifiermanual.pdf

Re: The Problem with Friendly C

#76

Post by praptak »

Fun fact about large shifts being undefined.

With a 32 bit x, the expression x > (32 - b) can be translated to a single "roll x b bits to the left" instruction. But it is only possible if >32 shifts are treated as undefined.

Re: The Problem with Friendly C

#77

Post by nickodell »

rswier wrote: post #63
pcwalton wrote: post #56

Earlier quoted context omitted.

> I don't believe C should be a language where the compiler does all sorts of high-level optimisation; it should be a straightforward "do what I say" type of language where you get almost exactly what you write, and the only optimisations should be at the level of things like instruction selection --- the optimisations that a programmer would not be able to do at the source level. In that case, you're asking for easi…

IMHO aggressive optimization at compile time is an example of premature optimization. Let the hardware have access to a straightforward representation. Once the run-time hot-spots are identified, the hardware (firmware, VM, whatever) can rewrite the binary code to execute faster. Excessive compiler optimization makes this difficult or impossible (too much information thrown away.) Compilers should be designed for fas…

It doesn't really seem like this solves the problem of optimizers introducing bugs and vulnerabilities.

Take the canonical optimizer-created security hole: the hardware optimizer replaces a constant-time compare (which doesn't leak timing information) with a variable-time compare (which does).

I don't think this solves the problem we're setting out to solve, ie. the optimizer introducing bugs.

Re: The Problem with Friendly C

#78

Post by Dylan16807 »

caf wrote: post #74
Dylan16807 wrote: post #35

Earlier quoted context omitted.

How about treating it just like another thread accessing the values? So yes in the general case, no if you put in memory barriers or atomics.

Race conditions are generally treated as "you get undefined behavior" for much the same reasons.

I mean race conditions on actual hardware, not the way the standard abandons all hope yet again.

Stale reads don't require total chaos.

Re: The Problem with Friendly C

#79

Post by barrkel »

pcwalton wrote: post #43
to3m wrote: post #3

If ((uint32_t)x But why can't it just produce something different on each system? Allow me to call it as I see it: the modern interpretation of undefined behaviour is bullshit. What compilers do today should be the recourse of absolute last resort, and the sort of thing that makes its authors feel bad. But it seems to be treated as a matter of course. I don't know what to say. Mandatory reading: http://robertoconcert…

> But why can't it just produce something different on each system? It could. That is indeed how Rust, for example, defines it. But in C "unspecified values" have a way of turning into undefined behavior really quickly. Here's an actual bug we have in Rust [1]: let index = 1.04E+17 as u8; let array = vec![1, 2, 3, 4, 5]; println!("{}", array[index as usize]); // segfault Why does that segfault instead of emitting a s…

The problem here is, as you point out, the conversion of double to unsigned byte.

More specifically, the problem is that processing this operation is not producing error; it instead propagates the problem and produces invalid code. That's a situation where nobody wins.

I think we'd be better off if many undefined behaviours were instead implementation defined. If, instead, LLVM had converted the double to some integer and then chosen the lowest 8 bits from that integer, we'd have gotten some random value - but at least the comparison wouldn't have been ignored. The comparison should only be ignored if the compiler can prove that the value definitely lies within the range; if it isn't sure, it can't remove the comparison. The programmer wrote it for a reason; ignoring the intent of the programmer is a bug.

On strict aliasing, I'm against it without explicit opt-in over a delimited subset of source code. I understand that using & is going to harm the performance of my code; I think that's an acceptable tradeoff for more predictable behaviour. If it harms the performance of C++ vector iterators, for example, I don't care: strict aliasing rules are a worse cure than the disease of C++'s poorly thought out abstraction tools. Let a subset of the program follow Fortran rules if it's required.

Re: The Problem with Friendly C

#80

Post by sanxiyn »

aplorbust wrote: post #67

Being an ignorant fool with an uninformed opinion, I would like to see a C compiler that is evaluated and critiqued not only based on the warnings and errors it generates but, more importantly, on the assembly it generates. Namely, how compact and readable is the generated asm? When we read the asm, can we easily follow what the compiler has done and _why_? As an ignorant fool, in my mind C is still a shorthand for w…

I agree it would be an interesting exercise to write a compiler optimized for assembly readability. But I don't think there is enough demand for such a thing so that it will get written.

It also would be interesting if those who think there is enough demand start a crowdfunding campaign to prove that there is enough demand.

Post reply on HN