Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Undefined behavior in C is a reading error

yodaiken.com

Post reply on HN
Print view View original

351–360 of 503 posts

Re: Undefined behavior in C is a reading error

#351

Post by a1369209993 »

hnjst wrote: post #344
a1369209993 wrote: post #340

Earlier quoted context omitted.

> I don't get what you or the comment you're responding to are wishing for. Quoting from another of my comments: > > [What are you objecting to?] > Inferring any propositional statement about the program (eg "this pointer is not null") from the fact that its negation would imply undefined behaviour. That is what the problem is. Undefined behaviour is a licence to implement operations without regard for unusual corner…

> Inferring any propositional statement about the program (eg "this pointer is not null") from the fact that its negation would imply undefined behaviour. I'm not completely sure I understand that correctly, but do you mean that statements that are constant unless considering a possible (and "credible"?) implementation of UB shouldn't be fair-game for the compiler to optimize out? EDIT: I think I see a more tricky ca…

> Dereferencing a pointer further in the code shouldn't be a valid justification for optimizing out previous tests of it being null.

Not further. Anywhere. If the implementation wishes to rewrite pointer dereferences from `use(*p)` to:

  if(!p) abort();
  use(*p);
it may do so (undefined behaviour!), but if it chooses not to do so, it may not later pretend that it did, and remove a explict `if(!p)` that the programmer wrote. Given:

  use(*p);
  if(!p) return NOPE;
this is fine:

  if(!p) abort();
  use(*p);
  //if(!p) return NOPE; // unreachable because of if, not because of use
but not:

  use(*p);
  //if(!p) return NOPE; // CVE-20XX-XXXXX
The implementation can optimize based on information (like "p is not null") that is actually true (even if that's because it made it true), but not based on information it assumed was true on the basis that it counterfactually could have made it true (but didn't).

> it would prevent many classes of branch pruning.

Yes, that's the general idea.

Re: Undefined behavior in C is a reading error

#352

Post by tsimionescu »

bregma wrote: post #211
tsimionescu wrote: post #206

Earlier quoted context omitted.

I would note that the article is explicitly contesting the definition of UB that you are giving here (though you are absolutely right that this is the de facto definition used by all major compilers, and the commtitee). Basically the article is arguing that UB should be similar to Unspecified behavior - behavior that the implementation leaves up to the hardware and/or OS. I'm not sure where I fall to this issue, thou…

from ISO/IEC 9899:2011 "Programming Languages -- C" 3.4.3 1 undefined behavior behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements 2 NOTE Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner charac…

It's not a huge leap, but it still doesn't mean that UB by definition means that the compiler is allowed to assume UB doesn't happen. Allowing the compiler to assume this is one reasonable result of this definition (I give an example of how this reasoning works somewhere else), but it is not equivalent to the definition of UB, in principle.

Re: Undefined behavior in C is a reading error

#353

Post by tsimionescu »

Gaelan wrote: post #245
tsimionescu wrote: post #206

Earlier quoted context omitted.

I would note that the article is explicitly contesting the definition of UB that you are giving here (though you are absolutely right that this is the de facto definition used by all major compilers, and the commtitee). Basically the article is arguing that UB should be similar to Unspecified behavior - behavior that the implementation leaves up to the hardware and/or OS. I'm not sure where I fall to this issue, thou…

> and the commtitee I'd argue that for a document like the C standard, if there's a well-known intended meaning, that is the meaning of the document - any other interpretation is purely academic.

To some extent, though the committee has not moved to make this definition explicit in the standard (UB = behavior the compiler is free to assume can't happen), for one reason or another.

Re: Undefined behavior in C is a reading error

#354

Post by Too »

PeterisP wrote: post #289
Animats wrote: post #221

Most of this discussion revolves around integer overflow. Part of the problem is that most of the computer hardware is now twos-complement arithmetic. Programmers think of that as part of the language. It's not, for C. C has run, in the past, on - 36 bit ones complement machines (DEC and UNIVAC) - Machines with 7-bit "char" (DEC) - Machines with 9-bit "char" (UNIVAC, DEC) - Machines where integer overflow yields a pr…

Explicit bounds everywhere mean that those bounds need to be automatically checked every time instead of only where explicitly specified by the programmer. This leads to safer code at a nontrivial performance tradeoff, one which would not be acceptable for C.

You could imagine bounds being baked into the types and checked at compile time.

    int{0..10}  foo  = 4
    int{0..10}  bar  = 5
    int{0..10}  buzz = foo+bar                // ERR: 10+10 potentially > 10
    int{0..10}  boom = wrapping_add(foo, bar) // OK
    int{0..100} sum  = foo+bar                // OK
There are tools which can do this, for example Code Contracts in C#. It becomes rather tedious and verbose so is something usually only left for very safety critical code.

Re: Undefined behavior in C is a reading error

#355

Post by a1369209993 »

d110af5ccf wrote: post #343
a1369209993 wrote: post #341

Earlier quoted context omitted.

> where each operation is perfectly reasonable when considered on its own. No, it is not. For example, the transformation: int read_and_discard = *p; // vvvv int read_and_discard = *p; __unsafe_assume_always(p != 0); is not reasonable, since p is not, in fact, always nonnull.

Your examples are far too simplistic. Real world code is going to be far more complex and will tend to resist trivial analysis. For example, please explain how to prevent this without also (inadvertently) preventing the removal of unnecessary null checks when functions are inlined. What about an unnecessary null check that's hidden inside a macro? What about whole program LTO? A macro could be used in a variety of si…

> For example, please explain how to prevent this without also (inadvertently) preventing the removal of unnecessary null checks when functions are inlined. What about an unnecessary null check that's hidden inside a macro?

If a null check is unnecessary, that's because it is reachable only from the not-null side of some previous null check. The compiler can track that information just fine, using the same tools it uses to track false information derived pointer dereferences.

If:

  if(!p) abort();
  // we know p is non-null here, regardless of the dereference
  use(*p);
  MACRO(p);
expands to:

  if(!p) abort(); // first null check (this is relevant to optimizing out unnecessary null checks)
  use(*p); // pointer dereference (this isn't)
  if(!p) abort(); // second null check (unnecessary *because of the if*)
  utilize(*p); // more pointer dereference
the compiler can optimize out the second if (the "unnecessary null check" you refer to) based on the first if, regardless of whether the pointer dereference is even there.

Re: Undefined behavior in C is a reading error

#356

Post by ezekiel68 »

I find this article to be so refreshing. The whole "theater of security" circus around the over-blown interpretations of this term is nothing less than breathtaking in some of its applications. Though not about c, the saga of how the author of the actix web server (written in rust) was literally driven from his own project by virtual pitchfork-wielding issue reporters and redditors was a true disgrace. He dared to explore the optimization space in ways they did not approve of. Happy ending: he recovered his composure and created a new, competing project, named ntex, on his own terms.

Re: Undefined behavior in C is a reading error

#357

Post by IcePic »

vinkelhake wrote: post #166
rectang wrote: post #102

Earlier quoted context omitted.

> But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. The inflamed backlash should tell you just how damaging it is to impose silent failure on meticulously written, previously fine programs.

If the program was previously "fine" on version x.y.z of some compiler, then it is most likely still fine on it. That's the target that the program was written for. There's some disagreement on whether you can call a program "fine" that breaks after switching to a newer version, or a different compiler. I see a lot of programmers out there that unfortunately use the behavior of their code on whatever compiler they're…

Well, I can imagine a program having something where new C comments ( // ) makes it not divide, like:

a = 5 //* junk here */ 2 ;

I'm sure there are ioccc or underhanded C contest entries doing this to make code work differently on compilers based on if // is starting a comment line or not.

Sure it is an ugly way of writing stuff and you'd be hard pressed to find lots of real world traps like this, but when/if you did have code that "suddenly" miscompiles you might actually think your old code with an old compiler did work, and a new compiler for "the same" language breaks your program. I don't think everyone code base should need full rewrites ever time a new compiler comes out.

Re: Undefined behavior in C is a reading error

#358

Post by IcePic »

gpderetta wrote: post #193
rndgermandude wrote: post #178

Earlier quoted context omitted.

To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway. At this point instead of "optimizing" it could, and in my humble opinion absolutely should, blow up the compilation by generating an UB error, so people can fix their stuff. What about backwards compatibility in regards to a new compiler version deciding to issue errors on UB now? You don't have any guarante…

> To do UB "optimizations", the compiler first needs to figure out that there is an UB it can "optimize" anyway. That's not how compwillrs work. In fact in the general case it is impossible to figure out at compile time that "there is an UB". The compiler instead assumes as an axiom that no UB can ever happen and uses the axioms to prove properties of the code. These days if you want to catch UB, compile with -fsanit…

> These days if you want to catch UB, compile with -fsanitize=undefined-behaviour. The program wll then trap if UB is actually detected at runtime.

So, let me get this straight, someone wants to make sure pointer p is not null (in the wrong way), and codes something like the examples in posts above like if (!p) ... and if that doesn't trigger calls use(*p), but compiler decides p can never be null because that would be UB and hence removes the check.

The C coder dumps the code and gets upset because the check is removed and gets the hint to catch UB by adding -fsanitize .. that "catches UB" in the above scenario so that the program will "trap if UB is detected".

I think we just came full circle there.

Sure, the -f will catch ALL detected bugs and so on, but I still found it a bit funny.

Re: Undefined behavior in C is a reading error

#359

Post by thaumasiotes »

hnjst wrote: post #329
a1369209993 wrote: post #313

Earlier quoted context omitted.

> people who think that silently optimizing away previously functional sanity checks is an acceptable engineering tradeoff To be fair, as several people and TFA have pointed out, this isn't a problem with C, but with defective/malicous C compilers . Admittedly, that's not much help if you can't find a compiler that isn't defective/malicous, though, so I can only wish you the best of luck.

I don't get what you or the comment you're responding to are wishing for. Do you want compilers to stop adding optimizations while staying within the bounds defined by the spec? That they somehow guess that a given piece of code that may trigger UB is too important for them to optimize it based on the assumption that the developer knew what she was doing and ensured that it wouldn't? The case of a compiler update bre…

> I fail to see why one would consider a compiler evolving while conforming to language specification defective or malicious

Conforming to the spec is not a virtue. We want the compiler to be reasonable, regardless of whether the spec is. When the spec is malicious, conforming to the spec is malicious behavior.

For example, the Java spec says that the This is absurd, and I'm comfortable calling it a bug in the spec. `a This behavior is documented, but that doesn't make things better, it makes them worse.

But the philosophy that says "if it's documented, then it's OK" doesn't even allow for the concept of a bug in the spec.

Re: Undefined behavior in C is a reading error

#360

Post by ndesaulniers »

Animats wrote: post #221

Most of this discussion revolves around integer overflow. Part of the problem is that most of the computer hardware is now twos-complement arithmetic. Programmers think of that as part of the language. It's not, for C. C has run, in the past, on - 36 bit ones complement machines (DEC and UNIVAC) - Machines with 7-bit "char" (DEC) - Machines with 9-bit "char" (UNIVAC, DEC) - Machines where integer overflow yields a pr…

I've heard of wrap around and saturate, but promote to float? What?! Do you have more info on how that worked?
Post reply on HN