Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every compiler writer should know about programmers (2015) [pdf]

complang.tuwien.ac.at

Post reply on HN
Print view View original

91–100 of 101 posts

Re: What every compiler writer should know about programmers (2015) [pdf]

#91

Post by pseudohadamard »

Doxin wrote: post #46
torben-friis wrote: post #38

Earlier quoted context omitted.

Sorry if I’m missing something as this isn’t my field, but shouldn’t the two meanings be roughly equivalent to the user? As in, everything down from UB is only working by an accident of implementation that does not need to hold, and you should explicitly not rely on that. Whether the compiler happens to explicitly make it not ever work or just leaves it to fate should not be relevant.

No, because the former definition is still something you can rely on given a specific compiler and a specific machine. Hell a bunch of UB was pretty much universal anyway. Compilers would usually still emit sensible code for UB. UB just ment "the spec doesn't define what happens". It didn't use to mean "the compiler can just decide to do any wild thing if your program touches UB anywhere at anytime". Hell, with the m…

One thing you need to add is that UB can be incredibly subtle and almost impossible to spot even by people with decades of programming experience. However, the compiler - and we're talking almost exclusively gcc here - will spot it and silently break your code. It won't warn "hey, I've spotted UB here!" even with every possible warning enabled, it will just quietly break your code without giving you any indication that it's done so.

It's some of the most user-hostile behavior I've ever encountered in an application.

Re: What every compiler writer should know about programmers (2015) [pdf]

#92

Post by mfru »

Panzerschrek wrote: post #80
mfru wrote: post #40

Earlier quoted context omitted.

Maybe Zig, Hare or C3 then?

Zig isn't a language I mean. It's still full of footguns. It doesn't address fundamental reliability issues of C.

I was responding to uecker, who had other preferences

Re: What every compiler writer should know about programmers (2015) [pdf]

#93

Post by pseudohadamard »

uecker wrote: post #89
zephen wrote: post #88

Earlier quoted context omitted.

A study found that, for a particular subset of UB (code that had legal, detectable behavior changes at differing optimization levels), 40% of Debian Wheezy packages exhibited this UB. https://people.csail.mit.edu/nickolai/papers/wang-stack.pdf I submit that that's a small fraction of UB, that much of it would exist at any optimization level.

I know, but this still leaves 60% of programs without such UB which is far from "it is practically impossible find a program without UB". Also this this was a study from 2013 and many of those bugs found were fixed. Also GCC got UBSan in 2013 (so after this study).

That's "UB that was detected in this study". Since gcc will silently break code when it detects UB and you can't tell until you hit that specific case, the 40% is a lower bound. In practice it could be anything up to the full 100%.

Re: What every compiler writer should know about programmers (2015) [pdf]

#94

Post by uecker »

pseudohadamard wrote: post #90
uecker wrote: post #33

Earlier quoted context omitted.

And please provide feedback to WG14. Also please give feedback and file bugs for GCC / clang. There are users of C in the committee and we need your support. Also keeping C implementable for small teams is something that is at risk.

Myself and other developers I know have tried giving feedback for gcc. On the whole, going outside and shouting at clouds is more productive.

I felt the same. There are too few contributors for GCC. At some point I started to fix the bugs that I had filed myself. Still, it is important that the user make themselves heard.

Re: What every compiler writer should know about programmers (2015) [pdf]

#95

Post by uecker »

pseudohadamard wrote: post #93
uecker wrote: post #89

Earlier quoted context omitted.

I know, but this still leaves 60% of programs without such UB which is far from "it is practically impossible find a program without UB". Also this this was a study from 2013 and many of those bugs found were fixed. Also GCC got UBSan in 2013 (so after this study).

That's "UB that was detected in this study". Since gcc will silently break code when it detects UB and you can't tell until you hit that specific case, the 40% is a lower bound. In practice it could be anything up to the full 100%.

In theory. But most C programs do not rely on UB. What is the basis for your claim?

Re: What every compiler writer should know about programmers (2015) [pdf]

#96

Post by pseudohadamard »

uecker wrote: post #95
pseudohadamard wrote: post #93

Earlier quoted context omitted.

That's "UB that was detected in this study". Since gcc will silently break code when it detects UB and you can't tell until you hit that specific case, the 40% is a lower bound. In practice it could be anything up to the full 100%.

In theory. But most C programs do not rely on UB. What is the basis for your claim?

Uhh... mathematics and logic? Since there's no perfect UB detector, one that detects UB in 40% of programs can only be presenting a lower bound. And I don't know why you think C programs rely on UB, they have it present without the programmer knowing about it.

Re: What every compiler writer should know about programmers (2015) [pdf]

#97

Post by pseudohadamard »

uecker wrote: post #94
pseudohadamard wrote: post #90

Earlier quoted context omitted.

Myself and other developers I know have tried giving feedback for gcc. On the whole, going outside and shouting at clouds is more productive.

I felt the same. There are too few contributors for GCC. At some point I started to fix the bugs that I had filed myself. Still, it is important that the user make themselves heard.

I think it's a circular problem, the gcc developers are very insular and respond to outside input with anything from ignoring it to getting into long lawyeristic arguments why, if you squint at the text just right, their way is the only right way, which strongly discourages outside contributions. There's only so many hours in the day and arguing till you're blue in the face that silently mutating a piece of code into unexpected different code that always segfaults when run based on a truly tortured interpretation of two sentences of text gets old fast. The gcc devs would make great lawyers for bypassing things like environmental law, they'd find some tortuous interpretation of an environmental protection law that let them dump refinery waste into a national park and then gleefully do it because their particular interpretation of the law didn't prohibit it.

Contrast this with Linus' famous "we do not break userspace" rant which is the polar opposite of the gcc devs "we love to break your code to show how much cleverererer than you we are". Just for reference the exact quote, https://lkml.org/lkml/2012/12/23/75, is:

  And you *still* haven't learnt the first rule of kernel maintenance?  If a change results in user programs breaking, it's a bug in the kernel. We never EVER blame the user programs. How hard can this be to understand?  ... WE DO NOT BREAK USERSPACE!
Ah, Happy Fun Linus. Can you imagine the gcc devs ever saying "if we break your code it's a problem with gcc" or "we never blame the user?".

This really seems to be gcc-specific problem. It doesn't affect other compilers like MSVC, Diab, IAR, Green Hills, it's only gcc and to a lesser extent clang. Admittedly this is from a rather small sample but the big difference between those two sets that jumps out is that the first one is commercial with responsibilities to customers and the second one isn't.

Re: What every compiler writer should know about programmers (2015) [pdf]

#98

Post by uecker »

pseudohadamard wrote: post #96
uecker wrote: post #95

Earlier quoted context omitted.

In theory. But most C programs do not rely on UB. What is the basis for your claim?

Uhh... mathematics and logic? Since there's no perfect UB detector, one that detects UB in 40% of programs can only be presenting a lower bound. And I don't know why you think C programs rely on UB, they have it present without the programmer knowing about it.

It follows from mathematics and logic that "larger than 40%" could be 100%, but it does not follow that this is likely or reasonable to assume.

Re: What every compiler writer should know about programmers (2015) [pdf]

#99

Post by uecker »

pseudohadamard wrote: post #97
uecker wrote: post #94

Earlier quoted context omitted.

I felt the same. There are too few contributors for GCC. At some point I started to fix the bugs that I had filed myself. Still, it is important that the user make themselves heard.

I think it's a circular problem, the gcc developers are very insular and respond to outside input with anything from ignoring it to getting into long lawyeristic arguments why, if you squint at the text just right, their way is the only right way, which strongly discourages outside contributions. There's only so many hours in the day and arguing till you're blue in the face that silently mutating a piece of code into…

In my experience it is worse with clang that even more aggressively uses UB than GCC to optimize (and Chris Lattner in his famous blog post very much justified this line of thinking), and I have seen similar things with MSCV. I do not know about the others.

I think that GCC changed a bit in recent years, but I am also not sure that an optimizing compiler can not have the same policy as the kernel. For the kernel, it is about keeping API's stable which is realistic, but an optimizing compiler inherently relies on some semantic interpretation of the program code and if there is a mismatch that causes something to break it is often difficult to fix. It is also that many issues were not caused because they decided suddenly "let's now exploit this UB we haven't exploited before" but that they always relied on it but an improved optimization now makes something affect more or different program. This creates a difficult situation because it is not clear how to fix it if you don't want to roll back the improvement you spend a lot of time on and others paid for. Don't get me wrong, I agree the went to far in the past in exploiting UB, but I do think this is less of a problem when looking forward and there is also generally more concern about the impact on safety and security now.

Re: What every compiler writer should know about programmers (2015) [pdf]

#100

Post by pseudohadamard »

uecker wrote: post #99
pseudohadamard wrote: post #97

Earlier quoted context omitted.

I think it's a circular problem, the gcc developers are very insular and respond to outside input with anything from ignoring it to getting into long lawyeristic arguments why, if you squint at the text just right, their way is the only right way, which strongly discourages outside contributions. There's only so many hours in the day and arguing till you're blue in the face that silently mutating a piece of code into…

In my experience it is worse with clang that even more aggressively uses UB than GCC to optimize (and Chris Lattner in his famous blog post very much justified this line of thinking), and I have seen similar things with MSCV. I do not know about the others. I think that GCC changed a bit in recent years, but I am also not sure that an optimizing compiler can not have the same policy as the kernel. For the kernel, it…

Good point, yeah. I really want to like clang because it's not gcc but they have been following the gcc path a lot in recent years. I haven't actually seen it with MSVC, but I'm still on an old pre-bloat version of Visual Studio so maybe they've got worse in recent versions too.

I think a lot of the UB though isn't "let's exploit UB", it's "we didn't even know we had UB in the code". An example is twos-complement arithmetic, which the C language has finally acknowledged more than half a century after the last non-twos-complement machine was built (was the CDC 6600 the last one's-complement machine? Were most of the gcc dev even born when that was released?). So everyone on earth has been under the crazy notion that their computer used twos-complement maths which the gcc (and clang) devs know is actually UB and allows them to do whatever they want with your code when they encounter it.

Post reply on HN