Undefined behavior in C is a reading error • Hacker News Board

Re: Undefined behavior in C is a reading error

#181

Post by msbarnett » Thu, May 20, 2021, 8:08 PM UTC

_kst_ wrote: post #173

msbarnett wrote: post #143

Earlier quoted context omitted.

"for (int i=param; i < param + 16; i++) does not have a guaranteed loop count in the presence of undefined behavior" is true, but it's equally true that the C standard is quite clear that undefined behavior can be ignored, so we can validly treat "for (int i=param; i < param + 16; i++)" as if it were guaranteed to loop 16 times in all cases.

No, the C standard doesn't say that "undefined behavior can be ignored" (which would mean what, making it defined?). It says, "NOTE Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, ...". It doesn't say that the behavior can be ignored. It says that the undefinedness can be ignored. The implementation doesn't have to take notice of the fact that the behavior is unde…

"undefined behavior can be ignored" (meaning: the case where this could overflow need not be considered and can be treated as though it does not exist) vs "The implementation doesn't have to take notice of the fact that the behavior is undefined" strikes me as a distinction without a difference given that we land in exactly the same spot: the standard allows us to treat "for (int i=param; i > An implementation might define the behavior, but it's still "undefined behavior" as that term is defined by the ISO C standard.

The point where we seem to disagree (and the pedantry here is getting tiresome so I don't know that there's any value in continuing to go back and forth of on it) is that yes, it's undefined behavior by the ISO C standard. BUT, the ISO C standard also defines the allowable interpretations of and responses to undefined behaviour. Those responses don't exist "outside" the standard – they flow directly from it.

So it's simultaneously true that the standard does not define it and that the standard gives us a framework in which to give its undefinedness some treatment and response, even if that response is "launch angband" or, in this case, "act as if it loops 16 times in all cases".

Re: Undefined behavior in C is a reading error

#182

Post by Rusky » Thu, May 20, 2021, 8:12 PM UTC

vyodaiken wrote: post #167

Rusky wrote: post #39

Earlier quoted context omitted.

> The idea that UB is carte blanche for implementations to do whatever is an unintended consequence of the vague language of the standard. Whether or not this was originally intended, it's certainly become the way the standard is written and used today, so that's kind of beside the point. Further, this is not some new idea that arose from the C standard. It's a basic, core idea in both software engineering and comput…

One issue is that under the prevailing interpretation, the existing semantics is not reliable. You do not know when or if the compilers will take advantage of UB to completely change the semantics they are providing. That's not tenable.

That's not how it works. Taking advantage of UB doesn't change the semantics, it just exposes which behaviors were never in the semantics to begin with. Barring compiler or spec bugs, we do in principle know exactly when the compiler may take advantage of UB. That's the point of a document like the standard- it describes the semantics in a precise way.

To be fair, the existing semantics are certainly complex and often surprising, and people sometimes disagree over what they are, perhaps even to an untenable degree, but that's a very different thing from being unreliable.

Re: Undefined behavior in C is a reading error

#183

Post by rurban » Thu, May 20, 2021, 8:14 PM UTC

pif wrote: post #127

cygx wrote: post #95

Earlier quoted context omitted.

It's not so much about cleverness, but knowledge and vigilance. You first have to be aware of all the footguns, and then be careful not to let any of them slip through...

> You first have to be aware of all the footguns, Knowing your tools is part of being a professional. C is not for amateurs.

Then use such a tool, but don't call it C, rather -std=gnuc-opt11, which always knows better than the author, without any warning.

Call it randomC, unsuitable for professional programmers, but extremly suitable for benchmark games and managers. Who prefer to ignore pesty overflows, underflows, memset, memcpy, dereferencing NULL pointers and other rare cases.

Re: Undefined behavior in C is a reading error

#184

Post by OskarS » Thu, May 20, 2021, 8:17 PM UTC

Gibbon1 wrote: post #177

OskarS wrote: post #111

Earlier quoted context omitted.

Assuming you defined signed integer overflow to follow two’s complement rules (the only reasonable interpretation other than UB), it would still be a guaranteed loop count of 16. (EDIT: i’m a dumbass, this is obvs not true. disregard this paragraph) There’s an interesting thing to note with that example though: even if you did make signed integer overflow defined, that code is still obviously incorrect if param + 16…

The real problem is in a better world 'int' would be replaced by types that actually exhibit the correct behavior. for a loop counter you want an index type that will seg fault on overflow. If you think not having that check is worth it the programmer would need to tag it with unsafe. It's also problematic because it's size is defined as at least 16 bits. But programmers which means you should never use it to store a…

I’m not sure I agree. If signed overflow is UB, loops like this can be optimized the hell out of. The most obvious way would be to unroll it and eliminate the loop (and loop variable) entirely, but you can also do things like vectorize it, maybe turn it in to just a small number of SIMD instructions. The performance gains are potentially enormous if this is in a hot loop.

With your magic int that traps on overflow, you couldn’t do that if the compiler was forced to rely on that behaviour. This is exactly why signed overflow is UB in C, and I don’t think that’s an unreasonable case for a language like C.

To be clear, my point is that this program is incorrect if overflow happens regardless of whether overflow is UB or not. So you might as well make it UB and optimize the hell out of it.

Re: Undefined behavior in C is a reading error

#185

Post by lixtra » Thu, May 20, 2021, 8:17 PM UTC

jcranmer wrote: post #120

rectang wrote: post #102

Earlier quoted context omitted.

> But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. The inflamed backlash should tell you just how damaging it is to impose silent failure on meticulously written, previously fine programs.

> meticulously written, previously fine programs With relatively few exceptions, if your program hits undefined behavior, then your program was already doing something pretty wrong to begin with. Signed overflow is a poignant example: in how many contexts is INT_MAX + 1 overflowing to INT_MIN actually sane semantics? Unless you're immediately attempting to check the result to see if it overflowed (which is extremely…

> With relatively few exceptions, if your program hits undefined behavior, then your program was already doing something pretty wrong to begin with.

The author claims: “We have the absurd situation that C, specifically constructed to write the UNIX kernel, cannot be used to write operating systems. In fact, Linux and other operating systems are written in an unstable dialect of C that is produced by using a number of special flags that turn off compiler transformations based on undefined behavior (with no guarantees about future “optimizations”). The Postgres database also needs some of these flags as does the libsodium encryption library and even the machine learning tensor-flow package.”

So basically the programers of the most used C programs consider the C standard so broken that they force the compiler to deviate from it. (Or they are not able to do things right?)

Re: Undefined behavior in C is a reading error

#186

Post by bonzini » Thu, May 20, 2021, 8:18 PM UTC

magnostherobot wrote: post #176

cperciva wrote: post #170

Earlier quoted context omitted.

The C standard says very little about how those types work. In particular, you can cast a pointer to one of them and then cast back to a pointer -- but only if you cast the exact same value back, and the intptr values are not guaranteed to be in any way meaningful. In particular, casting a pointer to intptr_t, doing arithmetic on it, and casting back is not guaranteed to do anything useful. It almost certainly will,…

Do you have an example of a situation in which you'd want to cast the result of arithmetic intptr_t values to a pointer? The situations I can think of off the top of my head would be better done as arithmetic between pointers.

Arithmetic on pointers in turn is only defined if the pointers point within the same object (or right past the end of that object).

One example of using intptr_t would be going from a pointer passed to free() to a metadata block for the memory that must be freed.

Re: Undefined behavior in C is a reading error

#187

Post by rocqua » Thu, May 20, 2021, 8:20 PM UTC

adwn wrote: post #135

tom_mellior wrote: post #116

Earlier quoted context omitted.

The world doesn't end, but in the "int" case you get nice vector code and in the "unsigned int" case you get much less nice scalar code: https://gcc.godbolt.org/z/cje6naYP4

Yes, that is true. The proper way for a compiler to handle this, would be to add a single overflow check before the loop, which branches to a scalar translation of the loop. Most realistic code will need a scalar version anyway, to deal with the prolog/epilog of the unrolled loop for iteration counts that aren't multiples of the unrolling factor. Surely you agree that treating unsigned overflow differently from signe…

To a mathematician, wrapping 2^n+1 back to 0 is a lot more intuitive than wrapping 2^n to -2^n. Mathematically the two systems are largely equivalent. They are equivalent when considering addition and multiplication. Both implement arithmetic modulo 2^n+1.

However, the canonical representation of this system runs from 0 to 2^n+1. Hence, if you were going to make one kind of integer overflow, and not the other, C made the correct choice.

That leaves out the question of whether the difference between the two cases is significant enough to have a difference in how overflow works.

Re: Undefined behavior in C is a reading error

#188

Post by bonzini » Thu, May 20, 2021, 8:25 PM UTC

steveklabnik wrote: post #87

mjburgess wrote: post #54

Commenters here seem to be missing the core thesis of this article. It's not about what the standard literally means; it's about it's spirit -- and the reason for its spirit. The issue is "undefined behaviour" should never have been interpreted this extremely. The standard may be silent on how extreme, but it is implausible to suggest that the standard was actually written to enable this. Compiler writers for C! dism…

Personally, I think that arguing that those who define and implement a standard don’t understand one of the most fundamental aspects of said standard is going to be an uphill battle. You could argue that they’ve lost their way, and the article flirts with this, but the path forward is the hard part, and IMHO rings a bit hollow: it’s asserted that these rules aren’t needed for performance, but no evidence is given, an…

Different code depends on different optimizations. A loop on an int** might benefit a lot from aliasing optimizations, because the compiler will assume that a[i] will remain the same after writing to a[i][j]. Other code may not benefit at all.

Likewise that loop may not benefit from signed overflow; instead an initialization loop that, by way of multiple level of macros, ends up doing a[i]=b[i]*1000/100, might become twice as fast if signed overflow rules let the compiler rewrite the assignment as a[i]=b[i]*10.

Re: Undefined behavior in C is a reading error

#189

Post by joelkevinjones » Thu, May 20, 2021, 8:27 PM UTC

Two points:

The notion that compilers that encounter undefined behavior are allowed to generate any code they want is a new interpretation, for some value of "new". I can't remember the first time I encountered such an interpretation being used by compiler writers to justify something they wanted to do until sometime after 2000.

The notion that John Regehr has (quoted in the article) that undefined behavior implies the whole execution is meaningless is not supported by the language of either the C89 or C99 standard, at least by my reading. The C89 standard has a notion of sequence points. Wouldn’t all sequence points executed before undefined behavior is encountered be required to occur as if the undefined behavior wasn’t there? It would seem so:

From the C89 standard: 2.1.2.3 Program execution

The semantic descriptions in this Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant. Accessing a volatile object, modifying an object, modifying a file, or calling a function that does any of those operations are all side effects, which are changes in the state of the execution environment. Evaluation of an expression may produce side effects. At certain specified points in the execution sequence called sequence points, all side effects of previous evaluations shall be complete and no side effects of subsequent evaluations shall have taken place.

The C99 standard has nearly identical language:

5.1.2.3 Program execution 1 The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant. 2 Accessing a volatile object, modifying an object, modifying a file, or calling a function that does any of those operations are all side effects,11) which are changes in the state of the execution environment. Evaluation of an expression may produce side effects. At certain specified points in the execution sequence called sequence points, all side effects of previous evaluations shall be complete and no side effects of subsequent evaluations shall have taken place.

Re: Undefined behavior in C is a reading error

#190

Post by Lvl999Noob » Thu, May 20, 2021, 8:28 PM UTC

jcranmer wrote: post #20

It's easy to pick on undefined behavior in C when you focus on the more gratuitous undefined behaviors such as signed overflow or oversized shifts. I'm not certain why these are undefined behavior instead of implementation-defined, but my suspicion is that these caused traps on some processors, and traps are inherently undefined behavior. Instead, if you dislike undefined behavior, I challenge you to come up with wor…

I am not a compiler dev or a high skill coder so my opinion might not matter much but I'll still lay them out here.

> The first case is the compiler hint...

The compiler should refuse to compile if it comes upon such a case. Those hints are as much used by programmers as they are by the compiler. It should emit a warning and necessary checks + abort code if those hints can neither be proven nor disproven statically.

> The second case is uninitialized memory...

For the first example, unless the compiler knows that f() must always be true, it should give a compile time error. In case that it does know that f() is always true, it should still emit a warning (or give an error anyways).

> The third category is traps...

I am honestly not sure about this one. It would depend on the behaviour defined for multiple references / dereferences done simultaneously and type casting. I would still probably expect the compiler to give out warnings atleast.

Edit: language / grammar / typos