Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why undefined behavior may call a never-called function

kristerw.blogspot.com

Post reply on HN
Print view View original

31–40 of 183 posts

Re: Why undefined behavior may call a never-called function

#31

Post by lisper »

electrograv wrote: post #8
XCSme wrote: post #7

Very interesting, to me seems like a "compiler bug". The compiler should not automatically set the static pointer value if the function that sets it is never called. Anyway, I guess "undefined behavior" is really undefined and it means anything can happen, so as per specs it's not a bug. Ultimately it's the programmer's mistake for having undefined behavior in his code.

Unfortunately, "it's not a bug, it's a feature!" -- there are long-standing design choices in C/C++ where various circumstances are explicitly designed to yield "undefined" behavior where literally anything goes. I believe the original intent of these are to give the compiler/optimizer more room to speed up the executable. Edit: The 'undefined' clause here is due to invoking a function at address 0, rather than any l…

It is a bug, but it's a bug in the spec. Saying that a common mistake like dereferencing the null pointer is undefined and therefore your program can do anything is not useful behavior. The only sane design is for any attempt to dereference the null pointer to cause the program to signal an error somehow. Exactly how that happens can be left unspecified, but that it must happen cannot be unspecified in a sane design. I don't see how any reasonable person could possibly dispute this.

Re: Why undefined behavior may call a never-called function

#32

Post by vilhelm_s »

connorcpu wrote: post #27
userbinator wrote: post #16

I suspect this is another critical point: as NeverCalled may have been called from, for example, a global constructor in another file before main is run clang doesn't analyse across module boundaries --- even when it theoretically could --- so it doesn't know for certain that NeverCalled() is indeed never called. Throughout the years I've grown increasingly displeased at how compilers handle even trivial cases like t…

I assumed this was a side-effect of devirtualization. Obviously indirections are slower, so if the compiler can look at a dynamic call and realize that there's only 1 possible function it could be calling right there, that's a win. Only my 2c

Right, the original blog post also points this out.

Re: Why undefined behavior may call a never-called function

#33

Post by rcthompson »

tyingq wrote: post #4
csomar wrote: post #3

Earlier quoted context omitted.

One of the advantages of using Docker! You can safely run this code multiple times and play with it without affecting your machine.

Sure, but you could also replace "rm -rf /" with "id" or something else non destructive. Somebody is going to copy paste that snippet and have a bad day.

If someone blindly copies, pastes, compiles, and runs a random snippet of code from the internet that is specifically described as producing weird and unexpected behavior, then they deserve the harsh lesson they're about to learn.

Re: Why undefined behavior may call a never-called function

#35

Post by pcwalton »

_0ffh wrote: post #19
kps wrote: post #11

I have become convinced that the current screw-the-programmer interpretation of ‘undefined behaviour’ was not intended, or even imagined, by the original ANSI C committee. Within the committee's mandate to ‘standardize existing practice’, it was simply an acknowledgement that C compilers translated straightforward C code into straightforward machine code without adding safety checks, and that simple code might —​ in…

This, just so much this! I've been longing for a C compiler with a "sane optimizations only"-switch like forever. I'd gladly give up on the additional couple of per cent speed improvement obsessive compulsive compiler writers managed to eke out by ignoring the source codes' obvious intentions and defending it with "but technically it's undefined behaviour"!

Everyone posts a comment like this every time undefined behavior surprises someone. But the reality is that the reason why C remains alive is that compiler writers have managed to make it fast. It is not "a couple of percent": these kinds of optimizations can make an enormous difference when, for example, they're the difference between vectorizing a loop and not doing that.

Compiler writers are not "obsessive compulsive". They respond to the demands of their customers. Frequently, the reason why these optimizations exist is that someone filed a bug asking "why doesn't the compiler do this seemingly-obvious optimization?" Often, the only reason these seemingly-obvious optimizations work at all is by exploiting undefined behavior.

Re: Why undefined behavior may call a never-called function

#36

Post by lisper »

This is literally a religious argument. No sane person would consider this acceptable behavior if not for the fact that there is a holy text ("the standard") that says it's acceptable. Well, it's not acceptable. It is no more acceptable than, say, a car that explodes if you push the wrong button at the wrong time, which would be clearly unacceptable even if there were a document blessed by a standards committee that said otherwise. Faulty code can literally make things blow up in today's world, so there is literally (and I really do mean literally) no difference between these two scenarios. It is truly a sad reflection on the state of our profession that we are even spending time arguing about these things instead of fixing the standard so that the language it defines is actually useful for writing programs rather than just a source of material for games of intellectual one-upsmanship, to say nothing of myriad real-world problems. You'd think that decades of security breaches caused by buffer overflows would make people think, "You know, it's 2017. Maybe array dereferencing without bounds checks is a bad idea even if it does let my code run a little faster." Alas.

Re: Why undefined behavior may call a never-called function

#37

Post by pcwalton »

kps wrote: post #11

I have become convinced that the current screw-the-programmer interpretation of ‘undefined behaviour’ was not intended, or even imagined, by the original ANSI C committee. Within the committee's mandate to ‘standardize existing practice’, it was simply an acknowledgement that C compilers translated straightforward C code into straightforward machine code without adding safety checks, and that simple code might —​ in…

The original ANSI C committee had no idea about modern optimization pipelines. If people had continually pushed back against undefined behavior back then, there's a good chance that by 2017 the result would have been that C would be dead, replaced by a language that allows for modern optimization techniques.

Re: Why undefined behavior may call a never-called function

#38

Post by cesarb »

userbinator wrote: post #16

I suspect this is another critical point: as NeverCalled may have been called from, for example, a global constructor in another file before main is run clang doesn't analyse across module boundaries --- even when it theoretically could --- so it doesn't know for certain that NeverCalled() is indeed never called. Throughout the years I've grown increasingly displeased at how compilers handle even trivial cases like t…

> Since this is the entire program, it's trivial to see that it's not,

Is it really the entire program, in the presence of things like LD_PRELOAD and global constructors? What prevents a library loaded by LD_PRELOAD from calling NeverCalled() before main() starts?

Re: Why undefined behavior may call a never-called function

#39

Post by _0ffh »

staticassertion wrote: post #28
userbinator wrote: post #25

Earlier quoted context omitted.

How would this prevent me from returning a pointer to a stack frame that no longer exists, just for example? It doesn't, but the outcome would be predictable : the pointer will always be pointing there . I assume you mean by "stack frame that no longer exists" something like returning a pointer to a local variable; what that would do is return the address where the variable was --- the memory address still exists, so…

Is that really such an improvement?

Maybe you won't quickly find a practical and non-contrived application for this specific case, but there are plenty of others. UB breaking buffer overflow checks springs to mind.

Re: Why undefined behavior may call a never-called function

#40

Post by Avernar »

mikeash wrote: post #30
connorcpu wrote: post #27

Earlier quoted context omitted.

I assumed this was a side-effect of devirtualization. Obviously indirections are slower, so if the compiler can look at a dynamic call and realize that there's only 1 possible function it could be calling right there, that's a win. Only my 2c

Exactly. This isn't "fuck the programmer if he fucks up," it's "let's try to do really good optimizations." It's really nice to be able to use abstractions that cost nothing because the compiler is smart. In this particular case, you might have a function pointer that exists for future expansion, but which currently only ever holds one value. In a case like that, it's really nice if the compiler can remove the indire…

> It's really nice to be able to use abstractions that cost nothing because the compiler is smart.

But the compiler is not smart. It's screwing up in certain cases. In this example if it was smart it would have figured out that the value never was initialized.

> In this particular case, you might have a function pointer that exists for future expansion, but which currently only ever holds one value.

Then define it as a regular function for now. The fact that you only thought of one function that needs it means you're making abstractions before you really needed them. And if you need a second function soon you'll loose the speed of the optimization anyways. And you did profile it first to figure out that this one tiny optimization actually matters, right? :)

But let's say you really needed to do it that way for whatever reason. If the compiler was smart enough to warn you that it wasn't initialized you could have made an empty function and initialized it to that. Problem solved and the compiler would be free to optimize it away.

> In a case like that, it's really nice if the compiler can remove the indirection (and potentially go further and do clever things like inline the callee or do cross-call optimizations).

Sure. Do a full program optimization and figure out that the function to initialize the pointer was actually called. Then do all those clever optimizations. The issue is that the compiler writers want the benefits of the optimization without doing the work making the optimization safe by making the compiler smarter. They just hide behind the "undefined behavior" mantra and let the programmer pick up the pieces when it goes wrong.

> For this particular scenario, the language should encode the nullability of Do as part of the type. If it's non-nullable, then it should require explicit initialization.

This. I 100% agree that this is the proper solution. But it would require a whole program pass to figure out that it's actually initialized somewhere. As I said above, the compiler writers could have done that without a change to the language.

But a lot of UB could be avoided by language changes. That's what many people have done when designing new languages. With C however we're stuck with what we have and need to make the compiler smarter before it slaps every optimization in its tool belt at every piece of code.

Maybe the C language needs to slowly evolve and add those changes to start getting rid of UB. But there has been zero progress in that direction. The compiler writers are perfectly content to squeeze out every last cycle of performance using any new UB loophole they can find.

When safety finally becomes a priority to them over benchmarks then maybe we'll start seeing some progress.

Post reply on HN