Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Undefined behavior in C is a reading error

yodaiken.com

Post reply on HN
Print view View original

111–120 of 503 posts

Re: Undefined behavior in C is a reading error

#111

Post by OskarS »

anarazel wrote: post #45
_kst_ wrote: post #31

The author suggests that the text following the definition of "undefined behavior", listing the permitted or possible range of undefined behavior, should be read to restrict the consequences. But the first possibility listed is "ignoring the situation completely with unpredictable results". Surely that covers any possible consequences. The author also says: > Returning a pointer to indeterminate value data, surely a…

> For example, if signed integer overflow yielded an unspecified result rather than causing undefined behavior, I wonder if any implementations would be adversely affected. I suspect so - makes it harder to reason about loop counts because the compiler can't necessarily guarantee that an incremented loop counter won't become negative and thus the loop needs to iterate more. E.g. something like for (int i=param; i Tha…

Assuming you defined signed integer overflow to follow two’s complement rules (the only reasonable interpretation other than UB), it would still be a guaranteed loop count of 16. (EDIT: i’m a dumbass, this is obvs not true. disregard this paragraph)

There’s an interesting thing to note with that example though: even if you did make signed integer overflow defined, that code is still obviously incorrect if param + 16 overflows. Like, the fact that signed integer overflow is UB is totally fine in this example: making it defined behavior doesn’t fix the code, and if making it UB allows the compiler to optimize, then why not?

Arguably, this is the case with the vast majority of signed integer overflow examples: the UB isn’t really the issue, the issue is that the programmer didn’t consider overflow, and if overflow happens the code is incorrect regardless. Why cripple the compilers ability to optimize to protect cases which are almost certainly incorrect anyway?

Re: Undefined behavior in C is a reading error

#112

Post by bhk »

msbarnett wrote: post #93
bhk wrote: post #68

Earlier quoted context omitted.

I've seen a real-world example something like this: int a[32] = {...}; int flag = 1 The "1 undefined behavior (!) when index is greater than 32 (on a platform with 32-bit integers), even if the result is never used! The compiler inferred that index must always be less than 32, which allowed it to optimize out the array bounds check, which turns the code into a write-anywhere gadget. Note that if the standard had not…

> Note that if the standard had not declared "n But also note that a lot of existing code doing bitshift-and-index inside a hot loop that never went out of bounds would now get slower if it started having to run bounds checks it had previously elided in an optimization pass. Let's not pretend that "it results in some implementation-specific value, or maybe traps" is a clear win with no downsides that Standards Author…

It isn't clear to me precisely what example you have in mind.

If you are saying that deleting array bounds checks might have performance benefits that outweigh the security concerns, then I disagree.

If you are saying that the compiler would have to insert bounds checks, I don't see how you arrive at that.

I have seen claims that gratuitous UB is important for enabling meaningful optimizations, but in every such case the examples did not hold up to scrutiny. In the end, the same optimization remains possible without the gratuitous UB, although it might involve a little more work on the part of the compiler engineer.

Regarding "malice": "Never attribute to malice..."

Re: Undefined behavior in C is a reading error

#113

Post by adwn »

msbarnett wrote: post #79
rectang wrote: post #43

Earlier quoted context omitted.

When compiler writers have get "creative" with C undefined behavior, programming C no longer produces predictable results. > least disruptive Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? I wish I could just fence off this insanity by never starting another project in C. Unfortunately, C is…

> Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? Yeah. Not doing that on modern processors is actually quite disruptive. Here: for(i = offset; i What C compilers currently do is, in line with the standard, ignore the case that offset + 16 might overflow. This makes this eligible for loop unr…

> If, instead, we force compilers to think about the fact that offset + 16 could have some implementation-defined meaning like wrapping, then all bets are off & we have to throw a bunch of optimization opportunities out the window.

Uh huh. If `i` is declared as `unsigned int` instead of `int`, then overflow is defined and the compiler can't apply those optimizations. And yet the world doesn't end and the sun will still rise tomorrow...

Re: Undefined behavior in C is a reading error

#114

Post by tom_mellior »

DangitBobby wrote: post #106

It truly boggles my mind that this discussion (linked in TFA) played out the way it did. In my mind it's 100% not okay to (by default) optimize away a check for integer overflow. I've never really written in C (or any unsafe language) before so I had little context for the types of traps C sets for developers. Based on the responses of the person who presumably implemented the optimization, it comes as no surprise th…

> In my mind it's 100% not okay to (by default) optimize away a check for integer overflow.

But a + 100 > a is not a check for overflow. If a >= INT_MAX - 100, it is an overflow. The "will this operation overflow" check would be a >= INT_MAX - 100, and GCC would not optimize that away.

Re: Undefined behavior in C is a reading error

#115

Post by [deleted] »

msbarnett wrote: post #58
setr wrote: post #32

Earlier quoted context omitted.

I believe he’s arguing that they were more sane/pragmatic compilers — they would be inherently less comfortable exploiting UB to do anything other than what people expected or were used to, because there is more real possibility of retribution (GCC could get away with making demons fly out your nose and just lose marketshare [that it doesn’t directly depend on anyways] where a commercial compiler would go out of busi…

> I believe he’s arguing that they were more sane/pragmatic compilers I can’t imagine they have any actual experience with Borland or Symantec’s C or C++ compilers, then. These things had notoriously shakey standards compliance and their own loopy implementations of certain things, along with legions of bugs – it’s not hard to find older C++ libs with long conditional compilation workarounds for Borland’s brain damag…

[deleted]

Re: Undefined behavior in C is a reading error

#116

Post by tom_mellior »

adwn wrote: post #113
msbarnett wrote: post #79

Earlier quoted context omitted.

> Like starting to optimize away loop checks that can "never happen" because signed integer overflow is UB, suddenly changing the behavior of programs that were fine for years? Yeah. Not doing that on modern processors is actually quite disruptive. Here: for(i = offset; i What C compilers currently do is, in line with the standard, ignore the case that offset + 16 might overflow. This makes this eligible for loop unr…

> If, instead, we force compilers to think about the fact that offset + 16 could have some implementation-defined meaning like wrapping, then all bets are off & we have to throw a bunch of optimization opportunities out the window. Uh huh. If `i` is declared as `unsigned int` instead of `int`, then overflow is defined and the compiler can't apply those optimizations. And yet the world doesn't end and the sun will sti…

The world doesn't end, but in the "int" case you get nice vector code and in the "unsigned int" case you get much less nice scalar code: https://gcc.godbolt.org/z/cje6naYP4

Re: Undefined behavior in C is a reading error

#117

Post by jerf »

dfabulich wrote: post #56

I think the problem is cultural in the C community. C programmers have Stockholm Syndrome around UB optimizations. As TFA notes, "There is No Reliable Way to Determine if a Large Codebase Contains Undefined Behavior" https://blog.llvm.org/2011/05/what-every-c-programmer-should... That's because UB is a bug that occurs as your program runs (e.g. dereferencing a null pointer). You'd have to prove that your program is f…

I expect there's already been a lot of evaporative cooling in the C community and there will only be more over time. Increasingly the people who are going to be left in the C community are precisely those people who think C is generally OK. Anyone who has developed a high degree of fear of C is ever-increasingly doing what it takes to move away.

I have tried to phrase this neutrally, as to not be a comment about whether or not C "really is" generally OK. I am not sure I have 100% succeeded, as I have my opinions. But the point stands regardless; the people in the C community are generally going to be the ones who are OK with C.

Re: Undefined behavior in C is a reading error

#118

Post by mjw1007 »

anarazel wrote: post #76

First: I do dislike how hard it is to avoid some UB / how impractical some of the rules are. But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. Almost describing them to write optimization passes looking for UB so they can over-optimize something, while cackling loudly in glee about all the programs they can break. The set of people doing so overlaps with the set…

I agree.

I don't have much sympathy for people who were doing things like writing multithreaded programs in the days before C documented its memory model and then becoming unhappy because new optimisations that legitimately help single-threaded code broke their programs.

In my experience C compiler maintainers have generally been open to the idea of offering guarantees beyond a narrow reading of the standard, but they want to be able to clearly state what it is that they're guaranteeing. "Keep old programs running" isn't enough.

I think the "Prevailing interpretation" that Yodaiken complains about is coming from the same place as suspicion of the "be lenient in what you accept" IETF principle: that sort of thing doesn't lead to robustness in the long run.

The way forward at this point is surely to define more things that are currently undefined (whether in the standard or by widely-implemented extensions).

Re: Undefined behavior in C is a reading error

#119

Post by jerf »

optymizer wrote: post #70
dfabulich wrote: post #56

I think the problem is cultural in the C community. C programmers have Stockholm Syndrome around UB optimizations. As TFA notes, "There is No Reliable Way to Determine if a Large Codebase Contains Undefined Behavior" https://blog.llvm.org/2011/05/what-every-c-programmer-should... That's because UB is a bug that occurs as your program runs (e.g. dereferencing a null pointer). You'd have to prove that your program is f…

Anecdotally, I find about as many bugs in my Java code at work, as I do in my hobby C code, including "dereferencing null pointers" aka NullPointerExceptions.

"Undefined behavior" is a term of art relevant to C, meaning that the standard no longer has any comment about what happens. Thus the comments about launching nukes, or destroying your machine, etc., being standards complaint, even though obvious real compilers won't actually emit code that does that.

Dereferencing a nil pointer in Java is not undefined behavior. It is defined; it throws a NullPointerException, and "it throws a NullPointerException" is a very specific term of art in the Java community that is very precisely defined.

Many languages don't have an explicit concept of "undefined behavior" in their standard (though, of course, due to deficiencies in the standard they may have implicitly undefined behavior), and of those that do I doubt anyone matches C or C++ in the commonly-used category of languages.

Re: Undefined behavior in C is a reading error

#120

Post by jcranmer »

rectang wrote: post #102
anarazel wrote: post #76

First: I do dislike how hard it is to avoid some UB / how impractical some of the rules are. But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. Almost describing them to write optimization passes looking for UB so they can over-optimize something, while cackling loudly in glee about all the programs they can break. The set of people doing so overlaps with the set…

> But I also think a lot of discussions of this topic caricaturize compiler writers to a ridiculous degree. The inflamed backlash should tell you just how damaging it is to impose silent failure on meticulously written, previously fine programs.

> meticulously written, previously fine programs

With relatively few exceptions, if your program hits undefined behavior, then your program was already doing something pretty wrong to begin with. Signed overflow is a poignant example: in how many contexts is INT_MAX + 1 overflowing to INT_MIN actually sane semantics? Unless you're immediately attempting to check the result to see if it overflowed (which is extremely rare in the code I see), this overflow is almost certain to be unexpected, and a program which would have permitted this was not "previously fine" nor "meticulously written."

I feel compelled right now to point out that software development is a field where it is routine to tell users that it's their fault for expecting our products to work (that's what the big all-caps block of every software license and EULA says, translated into simple English).

Post reply on HN