Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Undefined behavior in C is a reading error

yodaiken.com

Post reply on HN
Print view View original

151–160 of 503 posts

Re: Undefined behavior in C is a reading error

#151

Post by msbarnett »

bhk wrote: post #134
msbarnett wrote: post #79

Earlier quoted context omitted.

> Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? Yeah. Not doing that on modern processors is actually quite disruptive. Here: for(i = offset; i What C compilers currently do is, in line with the standard, ignore the case that offset + 16 might overflow. This makes this eligible for loop unr…

Well-defined integer overflow would not preclude loop unrolling in this case. One simple alternative would be for the compiler to emit a guard, skipping unrolling in the case that (offset+16) overflows. This guard would be outside the unrolled loop. Furthermore, unsigned values are often used for indices (the unsigned-ness of size_t pushes programmers in that direction) and unsigned overflow is well-defined, so any c…

> Well-defined integer overflow would not preclude loop unrolling in this case. One simple alternative would be for the compiler to emit a guard, skipping unrolling in the case that (offset+16) overflows.

To what end?

        for(i = offset; i
like most loops in most programs isn't designed to overflow. The program isn't any more correct for emitting two translations of the loop, one unrolled and one which is purely a bugged case anyways.

Changing the way the UB manifests while altering the nature of the optimization hasn't actually fixed anything at all here. All this would seem to accomplish would be to increase pressure on the icache.

Re: Undefined behavior in C is a reading error

#152

Post by wallstprog »

True story -- someone (not me) decided to initialize a C++ virtual class by defining a default ctor which does a "memset(this, 0, sizeof(*this))", then uses placement new to re-create the vptr.

Newer compilers complained more and more until gcc 8 which just silently ignored this awful hack -- no diagnostic, no error, just crickets.

Strictly speaking silently ignoring this atrocity is allowed by the standard, but it sure took a while to figure out. So be careful with code that "just works" even if it shouldn't.

Re: Undefined behavior in C is a reading error

#153

Post by btilly »

_kst_ wrote: post #31

The author suggests that the text following the definition of "undefined behavior", listing the permitted or possible range of undefined behavior, should be read to restrict the consequences. But the first possibility listed is "ignoring the situation completely with unpredictable results". Surely that covers any possible consequences. The author also says: > Returning a pointer to indeterminate value data, surely a…

The author suggests that the text following the definition of "undefined behavior", listing the permitted or possible range of undefined behavior, should be read to restrict the consequences.

But the first possibility listed is "ignoring the situation completely with unpredictable results". Surely that covers any possible consequences.

Absolutely not. In the C89 standard, undefined behavior becomes undefined *UPON USE OF* the thing that is undefined. In current compilers, the existence of undefined behavior anywhere in your program is an excuse to do anything that the compiler wants to with all of the rest of your program. Even if the undefined behavior is never executed. Even if the undefined behavior happens after the code that you have encountered.

So, for example, undefined behavior that can be encountered within a loop makes it allowable to simply remove the loop. Even if the undefined behavior is inside of an if that does not happen to evaluate to true with your inputs.

Re: Undefined behavior in C is a reading error

#154

Post by ynik »

_kst_ wrote: post #31

The author suggests that the text following the definition of "undefined behavior", listing the permitted or possible range of undefined behavior, should be read to restrict the consequences. But the first possibility listed is "ignoring the situation completely with unpredictable results". Surely that covers any possible consequences. The author also says: > Returning a pointer to indeterminate value data, surely a…

> For example, if signed integer overflow yielded an unspecified result rather than causing undefined behavior, I wonder if any implementations would be adversely affected.

You don't need to wonder. You can use -fwrapv to make signed integer overflow defined behavior.

C++20 introduced the guarantee that signed integers are two's complement. The original version of that proprosal also defined the behavior on overflow; but that part was rejected (signed integer overflow remains UB): http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p090... So at least the committee seems to think that the performance advantages are worth it.

Re: Undefined behavior in C is a reading error

#155

Post by _kst_ »

msbarnett wrote: post #145
_kst_ wrote: post #141

Earlier quoted context omitted.

No, the standard permits the implementation to ignore the behavior "with unpredictable results". If the value of param is INT_MAX, the behavior of evaluating param + 16 is undefined. It doesn't become defined behavior because a particular implementation makes a particular choice. And the implementation doesn't have to tell you what choice it makes. What the standard means by "ignoring the situation completely" is tha…

> that means the compiler can assume there's no overflow and generate code that always executes the loop body exactly 16 times Right. That's what I said. And just to be super-precise about the wording, the standard doesn't say "ignore the behavior 'with unpredictable results'" it says "Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results". Nitpicky, but the former wo…

And my point is that as far as the language is concerned, there is no guaranteed loop count under any circumstances. (An implementation is allowed, but not required, to define the behavior for that implementation.)

Re: Undefined behavior in C is a reading error

#156

Post by dooglius »

msbarnett wrote: post #79
rectang wrote: post #43

Earlier quoted context omitted.

When compiler writers have get "creative" with C undefined behavior, programming C no longer produces predictable results. > least disruptive Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? I wish I could just fence off this insanity by never starting another project in C. Unfortunately, C is…

> Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? Yeah. Not doing that on modern processors is actually quite disruptive. Here: for(i = offset; i What C compilers currently do is, in line with the standard, ignore the case that offset + 16 might overflow. This makes this eligible for loop unr…

> minority use-case

The amount of code that looks like this in a big enough hot loop to make a difference is negligible. Can you provide even one real-world example where this makes a difference, i.e. not some microbenchmark? The amount of code that can break as a result of signed overflows being UB, on the other hand, is huge.

> programmers would have to do a lot more fragile hand-unrolling of operations to get that performance back

Much easier ways to do this, e.g. by using an assert wrapper around __builtin_unreachable. Alternatively, an unsafe_int_t could be defined that gives the optimize-able behavior. The important thing is to make it opt-in; sensible defaults matter.

Re: Undefined behavior in C is a reading error

#157

Post by pklausler »

bhk wrote: post #68
jart wrote: post #42

> license for the kinds of dramatic and unintuitive transformations we’ve seen from the compilers, and any indication that undefined behavior should be a vehicle for permitting optimizations. Does anyone have an example of a time where Clang or GCC actually did something bad upon witnessing undefined behavior, rather than simply doing nothing, as the standard proposes? I ask because every time I've seen people get ma…

I've seen a real-world example something like this: int a[32] = {...}; int flag = 1 The "1 undefined behavior (!) when index is greater than 32 (on a platform with 32-bit integers), even if the result is never used! The compiler inferred that index must always be less than 32, which allowed it to optimize out the array bounds check, which turns the code into a write-anywhere gadget. Note that if the standard had not…

I think the compiler that you were using is broken. One can't infer "index use of "flag", and that inference can't override the predicates that dominate that use.

Re: Undefined behavior in C is a reading error

#158

Post by jart »

Jiro wrote: post #90
jart wrote: post #42

> license for the kinds of dramatic and unintuitive transformations we’ve seen from the compilers, and any indication that undefined behavior should be a vehicle for permitting optimizations. Does anyone have an example of a time where Clang or GCC actually did something bad upon witnessing undefined behavior, rather than simply doing nothing, as the standard proposes? I ask because every time I've seen people get ma…

This may not necessarily count as an example in the wild, but the 2013 Underhanded C contest at http://www.underhanded-c.org/_page_id_25.html includes this example: h = abs(h) % HASHSIZE; // Extra sanity check if (h = HASHSIZE) h = 0; return h; where h=INT_MIN causes the h to become negative and the sanity check is optimized out because abs(INT_MIN) is UB.

That's such a good example for teaching purposes, because it manages to combine the lack of understanding surrounding the remainder operator (i.e. unlike python % in c is not modulus) with the lack of understanding surrounding 0x80000000 (two's complement bane) into a single example. However it still makes my point that in this circumstance, the compiler's strategy still is to do nothing, because it can prove that the check could only be true under undefined behavior circumstances, so doing nothing means not compiling the check. I'm fine with that. Would anyone prefer that the compiler's internal definition of logic assume that absolute values can be negative? Must we throw out centuries of math because we've optimized integers to be 32-bits?

The only thing that's problematic is we need better tools to bring logic assumptions to our attention. Currently, UBSAN can only warn us about that when two's bane actually gets passed to the function at runtime. So the only way to spot faulty logic we failed to consider is to both enable UBSAN and be super methodical about unit testing.

Well, another thing I like to do is just read the assembly output. Constantly. Whenever I write something like a parser I've got a keyboard mapping that shows me the assembly output in Emacs with UBSAN enabled. If I turn off the noisy ones like pointer overflow then I can avoid these issues altogether by writing my code so that no UBSAN assembly gets generated. Since the compiler won't show that as a warning. You literally have to read the -S assembly output to get the compiler warnings that are actually meaningful.

Re: Undefined behavior in C is a reading error

#159

Post by anarazel »

jcranmer wrote: post #120
rectang wrote: post #102

Earlier quoted context omitted.

> But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. The inflamed backlash should tell you just how damaging it is to impose silent failure on meticulously written, previously fine programs.

> meticulously written, previously fine programs With relatively few exceptions, if your program hits undefined behavior, then your program was already doing something pretty wrong to begin with. Signed overflow is a poignant example: in how many contexts is INT_MAX + 1 overflowing to INT_MIN actually sane semantics? Unless you're immediately attempting to check the result to see if it overflowed (which is extremely…

> in how many contexts is INT_MAX + 1 overflowing to INT_MIN actually sane semantics? Unless you're immediately attempting to check the result to see if it overflowed (which is extremely rare in the code I see)

It's not that rare - I know that postgres got bit by particularly that issue, and several other other projects as well. Particularly painful because that obviously can cause security issues.

Re: Undefined behavior in C is a reading error

#160

Post by msbarnett »

dooglius wrote: post #156
msbarnett wrote: post #79

Earlier quoted context omitted.

> Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? Yeah. Not doing that on modern processors is actually quite disruptive. Here: for(i = offset; i What C compilers currently do is, in line with the standard, ignore the case that offset + 16 might overflow. This makes this eligible for loop unr…

> minority use-case The amount of code that looks like this in a big enough hot loop to make a difference is negligible. Can you provide even one real-world example where this makes a difference, i.e. not some microbenchmark? The amount of code that can break as a result of signed overflows being UB, on the other hand, is huge. > programmers would have to do a lot more fragile hand-unrolling of operations to get that…

> Can you provide even one real-world example where this makes a difference, i.e. not some microbenchmark

Sure. I don't even have to leave this thread to find one: https://news.ycombinator.com/item?id=27223954 reports a measurable speed impact to PostgreSQL when compiled with -fwrapv, which rules out the exact optimization in question.

This shouldn't be surprising; loops are extremely common and superscalar processors benefit enormously from almost anything other than a naïve translation of them.

Here's -fwrapv cutting performance of a function in half in Cython vs the non-fwrapv compilation: https://stackoverflow.com/questions/46496295/poor-performanc...

Post reply on HN