On C-optimizing compilers removing code that has undefined behavior • Hacker News Board

Re: On C-optimizing compilers removing code that has undefined behavior

#41

Post by yorwba » Sat, Jun 17, 2017, 11:09 AM UTC

srett wrote: post #26

andreyv wrote: post #5

> The programmer has clearly attempted to set x[0]=0 Then why not just write it like that? This is like deliberately shooting yourself in the foot and complaining that your shotgun works.

Still, making the compiler do whatever the hell it wants because "hey it's undefined behavior so we have the license to" is just as idiotic. Make the compilation fail and then add a flag to override for those who feel extra smart.

The compiler is not doing "whatever the hell it wants". It chooses an interpretation of the undefined behavior and then optimizes based on that. Of course, if the programmer had something else in mind, they will be surprised about the result; but it's their fault for not being precise enough.

So why make reads of uninitialized values undefined at all? Consider code like this:

  uint64_t x; // Will be initialized later
  /* ... */
  if(check_some_condition(y, z)) x = (uint8_t) f(b,d);
  /* ... */
  if(check_some_other_condition(foo)) x = (uint8_t) bar;
  /* ... */
  x &= 0xf;
  /* ... */
  if(x & 0x800) do_stuff();

Now the compiler has no way of knowing whether x will be initialized or not. But if it has been initialized, then it's value must be in 0..255, so the & 0xf can be limited to the lowest byte. But this means that the test for x & 0x800 will always be false, and stuff will never be done, so the compiler can optimize it out, reducing code size and thus cache pressure.

If these assumptions don't hold, then some later code may get passed a value for x that has the bit in 0x800 set, and expect do_stuff() to have initialized some data structure, which didn't happen, and the code blows up. But the compiler was just working from what it knew, and under the assumption that the programmer wouldn't depend on completely arbitrary values that happened to be in memory, everything it did was perfectly sensible.

Re: On C-optimizing compilers removing code that has undefined behavior

#42

Post by bluejekyll » Sat, Jun 17, 2017, 11:18 AM UTC

> Despite the best efforts of developers of rival programming languages

This is a flippant statement. It could be argued C++ falls into this category, but others are so new that they just haven't had the time to grow into the communities that would benefit from them.

> C’s advantages have preserved it as an indispensable systems and applications programming language.

But what about the disadvantages? I love C, I will always have fond memories of using it to do mind-bending things, but given the choice of using safe languages vs. an unsafe one at this point in time is a mistake.

Where there is an option to use a safe language instead of C, that option should always be chosen. It could potentially save lives depending on where it's deployed; it will definitely save money in the long term with fewer bugs.

C is not the only low cost abstraction and peak performance language available out there anymore.

Re: On C-optimizing compilers removing code that has undefined behavior

#43

Post by minipci1321 » Sat, Jun 17, 2017, 11:19 AM UTC

Maybe the commitee tries to kill the language. Or maybe it tries to divert from the field the people who routinely abuses the language and tries to outsmart the compiler. Those people will now have to try harder, and maybe at some point this additional time lost on debugging and validation, will start showing up in teams' stats. I say -- good thing.

Re: On C-optimizing compilers removing code that has undefined behavior

#44

Post by ajanuary » Sat, Jun 17, 2017, 11:21 AM UTC

clarry wrote: post #36

GedByrne wrote: post #14

Earlier quoted context omitted.

The problem is that the language allows this code and the compiler accepts it. If the code really does make no sense, then why not disallow it or capture it with a compiler warning? Instead the optimizer is to silently remove the code. Isn't it possible that the programmer knows something that the optimizer is unaware of?

> If the code really does make no sense, then why not disallow it or capture it with a compiler warning? You answered your own question: > Isn't it possible that the programmer knows something that the optimizer is unaware of? Here's a fun exercise. Find the best static analyzer you can. Run it on some substantial body of C code. Count all the false positives and false negatives (good luck). Now you should have an id…

C# is an example of a language that decided that including definite assignment analysis into the language spec was worth the tradeoff.

Re: On C-optimizing compilers removing code that has undefined behavior

#45

Post by DougBTX » Sat, Jun 17, 2017, 11:26 AM UTC

clarry wrote: post #36

GedByrne wrote: post #14

Earlier quoted context omitted.

The problem is that the language allows this code and the compiler accepts it. If the code really does make no sense, then why not disallow it or capture it with a compiler warning? Instead the optimizer is to silently remove the code. Isn't it possible that the programmer knows something that the optimizer is unaware of?

> If the code really does make no sense, then why not disallow it or capture it with a compiler warning? You answered your own question: > Isn't it possible that the programmer knows something that the optimizer is unaware of? Here's a fun exercise. Find the best static analyzer you can. Run it on some substantial body of C code. Count all the false positives and false negatives (good luck). Now you should have an id…

> Now you should have an idea as to why you can't simply require the compiler to reject the code and issue a diagnostic.

It is quite a different beast, but C# produces compile time failures when attempting to read from a potentially uninitialised variable. What is it that makes C different?

Re: On C-optimizing compilers removing code that has undefined behavior

#46

Post by xorblurb » Sat, Jun 17, 2017, 11:26 AM UTC

andreyv wrote: post #5

> The programmer has clearly attempted to set x[0]=0 Then why not just write it like that? This is like deliberately shooting yourself in the foot and complaining that your shotgun works.

That was just an example but I'm starting to have horror visions: imagine you have a struct with some gaps because of alignment, and you read it as chars (which you are supposed to can and must do to get the representation) after having properly initialized all its fields. You might hit an UB just by doing that, maybe not according to the standard for obscure reasons (I hope so), but the probability you are going to get that one day from a compiler developed by a moron^W zealot UB code deletionist is non-negligible (they do have misqualified some valid constructs as UB in the past...)

BTW if my example is actually valid, then that rule of reading uninitialized object being UB is some of the hugest shit, because depending of the type maybe reading an uninitialized byte will be UB, or maybe not, depending of how that byte was in a field of a type or just alignement... In practice, this is even more crazy, because it also depends on which parts of the code your compiler have seen, and it capabilities to break^W understand it, so you might as well considered it has a layer of non-determinism above all that.

C/C++ has become too dangerous. Use something safe instead.

Re: On C-optimizing compilers removing code that has undefined behavior

#47

Post by xorblurb » Sat, Jun 17, 2017, 11:31 AM UTC

yorwba wrote: post #41

srett wrote: post #26

Earlier quoted context omitted.

Still, making the compiler do whatever the hell it wants because "hey it's undefined behavior so we have the license to" is just as idiotic. Make the compilation fail and then add a flag to override for those who feel extra smart.

The compiler is not doing "whatever the hell it wants". It chooses an interpretation of the undefined behavior and then optimizes based on that. Of course, if the programmer had something else in mind, they will be surprised about the result; but it's their fault for not being precise enough. So why make reads of uninitialized values undefined at all? Consider code like this: uint64_t x; // Will be initialized later…

That understandable (but still dangerous). However, considering that reading an unsigned char can reasonably ever be an UB is insanely stupid, considering chars have a special place in the standard. There are going to be (new) problems because of that.

Re: On C-optimizing compilers removing code that has undefined behavior

#48

Post by xorblurb » Sat, Jun 17, 2017, 11:34 AM UTC

Karellen wrote: post #32

GedByrne wrote: post #14

Earlier quoted context omitted.

The problem is that the language allows this code and the compiler accepts it. If the code really does make no sense, then why not disallow it or capture it with a compiler warning? Instead the optimizer is to silently remove the code. Isn't it possible that the programmer knows something that the optimizer is unaware of?

The standard can't mandate a diagnostic for undefined behaviour, because it's not always possible to determine if behaviour is undefined. (Turing completeness) However, an implementation is allowed to issue any diagnostics it likes, so if you want one for this case, bug your compiler vendor to add one. Or, check the manual - it's possible that your compiler already does issue a diagnostic if you turn the right warnin…

> The standard can't mandate a diagnostic for undefined behaviour, because it's not always possible to determine if behaviour is undefined.

Yet that argument is stupid if we are talking about compilers issuing warnings for the UB they "detect", because by definition they have detected them. It might not be possible when it does not work exactly like that in some cases, but at least before removing some code, this is desirable.

It might also not be easy given the current internal design of compilers, but then I argue those design should be changed, because it is just too dangerous.

Re: On C-optimizing compilers removing code that has undefined behavior

#49

Post by xorblurb » Sat, Jun 17, 2017, 11:51 AM UTC

szemet wrote: post #25

Undefined behaviour was always there to make possible compilers to optimize as they like (nowadays usually for speed or to ease porting to specific architectures) - that is the essence of C "portable assembly" mentality and it always was. Am I wrong? I tolally agree that this is a bad mentality for software developement in general, but at least it is the "authentic C way" so I feel that part of the critics ungrounded…

Nope. The age of the sufficiently "advanced" (actually: retarded?) compiler was only theoretical during most of the lifetime of C, and UB were actually defined mostly because of differences between processors, NOT compilers. Compilers were faster thanks to their backend.

The standard even says: "undefined behavior: behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements"

However, today, you should act as if the nonportable part of this statement don't exist anymore, and act as if ALL UB are absolutely forbidden and result in the worst non-deterministic consequences, in all cases and regardless of your actual target. Meaning that today on e.g. x86, if you use a mainstream compiler, you ARE limited by some of the limitations of e.g. some obscure outdated DSPs.

And this is likely impossible to check in a non-trivial program.

So just use another language. C for serious purposes is dead. (some projects that started in that language continue to be developed in it, but given the security issues it creates, this will become unacceptable in a not too far future; so don't wait it is too late to switch: lead that movement.)

Re: On C-optimizing compilers removing code that has undefined behavior

#50

Post by szemet » Sat, Jun 17, 2017, 11:52 AM UTC

wfunction wrote: post #31

Uhm, I'm pretty sure this page is wrong, and the compiler cannot optimize this out. An "indeterminate" value is either an unspecified value or a trap representation. Like they said, the value here cannot be a trap representation, so it's just an unspecified value. An unspecified value is merely an unknown value, NOT a dynamically-mutating value, and NOT a trap representation. So XORing an unspecified value with itsel…

Here the first answer says (with some quotations) that it depends on, if the variable could be declared as register or not.

https://stackoverflow.com/questions/11962457/why-is-using-an...

edit: i recognised that it does not count as the original example is about an array element...