Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Do Not Know C: Short quiz on undefined behavior (2015)

kukuruku.co

Post reply on HN
Print view View original

161–170 of 189 posts

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#161

Post by to3m »

yorwba wrote: post #147
to3m wrote: post #127

Earlier quoted context omitted.

But the null pointer check was left in. (And, sure enough, adding -fno-delete-null-pointer-checks makes no difference.)

That's because y is never used, so why should the pointer be dereferenced? If you call bar(y) instead of just bar(), "x86-64 clang 3.9.1" with -O3 does the load as well (but after the check) foo(int*): # @foo(int*) test rdi, rdi je .LBB0_1 mov edi, dword ptr [rdi] jmp bar(int) # TAILCALL .LBB0_1: ret Only GCC does the kind of aggressive optimization the article mentions (and might need to be tamed by -fno-delete-null…

Ha... yes, a good point. That's a reasonable reason not to generate the load, and stupid me for not noticing. What's also dumb is that my eyes just glossed over the ud2 instruction that both put in main too. The program (not unreasonably) won't even run properly anyway.

gcc does seem to be keener than clang to chop bits out - I think I prefer clang here. But let's see how I feel if I encounter this in anger in a non-toy example ;) I must say I'm still a bit suspicious, but I can't really argue that this behaviour is especially surprising here, or difficult to explain.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#162

Post by E6300 »

vyodaiken wrote: post #160
E6300 wrote: post #143

Earlier quoted context omitted.

This? http://blog.regehr.org/archives/970 The compiler didn't create a security bug by removing the null check. The bug was created by the programmer when he didn't check for null before dereferencing the pointer. Even with the check, the program contained a bug.

The compiler converted a buggy program that was prevented from opening a security hole by defense in depth into a program with a security hole. It transformed a careless error into a systemic error, all in the cause of a micro-optimization that didn't.

What are you talking about? Dereferencing invalid memory is a security bug.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#163

Post by E6300 »

vyodaiken wrote: post #159
E6300 wrote: post #152

Earlier quoted context omitted.

In order to do that, the standard would have to define that dereferencing a null pointer must produce a deterministic behavior. There are only two possible behaviors: 1. The program successfully reads/writes that memory location and retrieves/overwrites whatever is there without crashing. Then the program can continue on and execute the if even if the pointer was null. 2. The program crashes immediate whenever a null…

C is not Haskell or Java. The C programmer may intend to interact with actual hardware and is not required to interact with some abstract machine. The standard can reflect this or it can attempt to convert C into a poorly designed high level language. Dereferencing the null pointer should be implementation dependent, but the compiler should be required to either detect and flag this as an error or compile it into the…

Sorry, but you are just wrong. The C standard does define an abstract machine.

> but the compiler should be required to either detect and flag [dereferencing the null pointer] as an error

How could the compiler detect at compile time the value of a run time variable? Sure, some instances might be detectable, but those are the extreme minority. Static analysis tools such as Clang are already capable of finding those compile time NULLs.

> or compile it into the machine operations indicated. The actual execution in the second case may depend on the environment.

Which is exactly what's done now. In most platforms accessing NULL causes a crash, so either the pointer is not null and the program doesn't crash, so the check in redundant; or the pointer is null and the does crash, so the check is never executed.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#164

Post by vyodaiken »

E6300 wrote: post #162
vyodaiken wrote: post #160

Earlier quoted context omitted.

The compiler converted a buggy program that was prevented from opening a security hole by defense in depth into a program with a security hole. It transformed a careless error into a systemic error, all in the cause of a micro-optimization that didn't.

What are you talking about? Dereferencing invalid memory is a security bug.

Not necessarily.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#166

Post by maxlybbert »

vyodaiken wrote: post #133
maxlybbert wrote: post #73

Earlier quoted context omitted.

> C programmers should be able to expect that "optimizations" will not transform program meaning. That's the official rule, but it's "program meaning as defined by the standard." It's not perfect, but nobody's come up with a better alternative. We get bugs because programmers expect some meaning that's not in the standard. But compilers are written according to the standard, not according to some folklore about what…

Actually that's not correct. The standard says the behavior is up to the compiler. The compiler author took that as a license to produce a non truth preserving transformation of the code. The actual current clang behavior also satisfies the standard as written.

> The standard says the behavior is up to the compiler.

I think this statement is correct, but it's the kind of thing people say when they confuse implementation defined behavior and undefined behavior. And that distinction is key.

Implementation defined behavior means the compiler gets to choose what it will do, document the choice, and then stick to it.

Undefined behavior means that the program is invalid, but the compiler isn't expected to notice the error. Whatever the compiler spits out is acceptable by definition. The compiler can generate a program that doesn't follow the rules of C; or that only does something weird when undefined behavior is triggered, but the weird behavior doesn't take place on the same line as the undefined behavior; etc.

It's certainly true that "the compiler isn't expected to notice the error" doesn't prohibit a compiler from noticing the error. A compiler can notice, but it's standard conforming even if it doesn't.

I should probably mention that when I say "the standard" I mean the C language standard; the POSIX standard may add requirements such that something undefined according to the language is well defined on POSIX.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#167

Post by dbaupp »

adamnemecek wrote: post #85
bcoates wrote: post #76

Earlier quoted context omitted.

It allows the optimizer to assume away the halting problem; all nontrivial loops are obligated to halt.

What's the point of this though? Why are you letting programmers write non-functional code? When does the loop exactly terminate? I'm guessing the standard discusses this but at this point idk if I care about memorizing more C trivia.

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1528.htm discusses it in some detail.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#168

Post by vyodaiken »

mattkrause wrote: post #86
vyodaiken wrote: post #58

Earlier quoted context omitted.

C programmers should be able to expect that "optimizations" will not transform program meaning. And because C is so low level, certain types of optimizations may be more difficult or impossible. If the pointer was explicitly set to NULL, the compiler can justifiably deduce the branch will not be taken but the deduction "if the programmer dereferenced the pointer it must not be NULL" is not based on a sound rule. In f…

> In fact, the whole concept that the compiler can make any transformation it wants in the presence of UB is wacky. That's the way it's often explained but it's not really what happens--the compiler doesn't scan for undefined behavior and then replace it with random operations. Instead, it's applying a series of transformations that preserve the program's semantics if the program stays "in bounds", avoiding invoking…

"touching naivete"is a good way of putting it.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#169

Post by vyodaiken »

E6300 wrote: post #165
vyodaiken wrote: post #164

Earlier quoted context omitted.

Not necessarily.

In what cases it's not?

In the referenced case the introduced error involved a reference to a null pointer but there was still no exploitable security hole. The exploit was enabled when the compiler removed an explicit check. The null dereference was an error, but it was not a security issue on its own.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#170

Post by E6300 »

vyodaiken wrote: post #169
E6300 wrote: post #165

Earlier quoted context omitted.

In what cases it's not?

In the referenced case the introduced error involved a reference to a null pointer but there was still no exploitable security hole. The exploit was enabled when the compiler removed an explicit check. The null dereference was an error, but it was not a security issue on its own.

Why didn't the kernel panic when it tried to access NULL?
Post reply on HN