“gcc will quietly break nearly half of all the packages that it compiles” • Hacker News Board

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#61

Post by quotemstr » Sat, Oct 24, 2015, 6:05 PM UTC

copsarebastards wrote: post #28

the_mitsuhiko wrote: post #20

Inknow people are quick to complain about programmers relying on UB here but this really is a long standing disagreement with the gcc folk. They are language lawyers of the worst sort and do not consider security implications being a point of discussion :(

> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. --Charles Babbage The modern version of this seems to be: "Mr. Babbage, I put the…

What GCC is doing subverts the purpose of writing in C, which is to get close to the machine and instruct its processor to do certain things. Optimizations are useful when they allow the compiler to better express our intent, but lately, we've seen compilers more and more often ignore our plain, stated intent, then back it up with a reference to a specification that wasn't intended to allow this subversion.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#62

Post by copsarebastards » Sat, Oct 24, 2015, 6:10 PM UTC

EdiX wrote: post #50

geofft wrote: post #23

Earlier quoted context omitted.

IMO, it's a little subtler than that. It's not the compiler's fault, it's the language's . Plenty of languages have no undefined behavior that can be written by accident. Go and safe Rust, for instance, have just about no undefined behavior at all, and are both performance-competitive with C. (Go has UB if you cause race conditions, and Rust has UB within `unsafe` blocks analogous to C's UB.) A C compiler, meanwhile,…

Yes, the C language specification is a bit shit, it leaves too much leeway to compilers so that C compiler for broken, niche architectures can be written. However, I disagree with you. All this badness in the C specification did not stop people from writing reasonable C compilers for reasonable architectures for decades. The real problem here is competition. Gcc is in a competition with clang to produce fast code whi…

> This is a case of following the letter of the law (in this case the C standard) while disregarding its spirit: all the undefined behaviour was so that C compilers could accomodate for odd architectures while remaining close to the metal, not so that compiler programmers could go out of their way to turn their compiler into a mine field.

Computers don't have spirits; they work as you tell them to work, to the letter, and if you're remaining close to the metal, your language will indicate that fact. Optimizing undefined behaviors doesn't make C a minefield; low-level programming for different architectures just is inherently a minefield. C was a minefield before these optimizations were added.

Rust is extremely impressive because they've found so many ways to do high-level programming while maintaining low-level performance. But they can only do that because they have the benefit of the 4 decades of programming language research that have occurred since the basics of C were designed.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#63

Post by copsarebastards » Sat, Oct 24, 2015, 6:15 PM UTC

quotemstr wrote: post #61

copsarebastards wrote: post #28

Earlier quoted context omitted.

> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. --Charles Babbage The modern version of this seems to be: "Mr. Babbage, I put the…

What GCC is doing subverts the purpose of writing in C, which is to get close to the machine and instruct its processor to do certain things. Optimizations are useful when they allow the compiler to better express our intent, but lately, we've seen compilers more and more often ignore our plain, stated intent, then back it up with a reference to a specification that wasn't intended to allow this subversion.

> What GCC is doing subverts the purpose of writing in C, which is to get close to the machine and instruct its processor to do certain things.

That's not the purpose of writing in C. It's a goal you might be able to achieve with C, but I'm not sure why that's your goal, and it's certainly not the goal of everyone who writes C. I think more people who write C do so with the coal of producing programs that run fast or with a minimal memory fingerprint.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#64

Post by MaulingMonkey » Sat, Oct 24, 2015, 6:17 PM UTC

copsarebastards wrote: post #53

makomk wrote: post #45

Earlier quoted context omitted.

The problem is that it's often perfectly clear, reasonable code on all the systems it was intended to run on. For example, on all Unix-like systems, pointer arithmetic is simply arithmetic and behaves like it. (C's predecessor didn't even have separate pointer and integer types.) So prior to compiler optimisations, this series of operations is safe and well-behaved on all architectures Linux supports even if a is NUL…

I understand the problem, I'm saying that it's not GCC's problem. If you don't want undefined behavior, don't put undefined behavior in your code. The code you wrote isn't clear or reasonable, because it relies undefined behavior. It's a valid criticism that this code does appear to be straightforward when it isn't, but that's not a criticism of GCC, it's a criticism of ANSI C. If you don't like it, use a better lang…

> If you don't want undefined behavior, don't put undefined behavior in your code.

I'd quip that this is statistically impossible for a sufficiently large codebase.

> it's a criticism of ANSI C. If you don't like it, use a better language.

This is my basic stance. However, if I'm e.g. in a situation where I have a C or C++ codebase I can't afford to rewrite from scratch, I'd like to use a "Better C" compiler, where "Better C" is a slightly less bad version of "ANSI C" - some undefined behavior removed, for example.

As shorthand, I'll generally refer to compilers for "Better C" as "Good C Compilers".

GCC is not trying to be a Good C Compiler. They've decided these things aren't their problem. Which is... fair. That's their choice. I do not for one minute pretend to understand that choice however - and it gives me yet one more reason to switch to a Good C Compiler.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#65

Post by overgard » Sat, Oct 24, 2015, 6:17 PM UTC

copsarebastards wrote: post #48

overgard wrote: post #44

Earlier quoted context omitted.

Compilers will do things like remove a memset clearing a chunk of memory to zero (because it detects that the variable isn't read again). That sort of thing is bad for security.

Writing code that depends on the value of unread memory is bad for security.

I think you misunderstood the example -- the memory is cleared after use to ensure that if it's reallocated by someone else, or someone hooks up a debugger, the content can't be examined (except when the compiler removes this clearing attempt because of an optimization). Lets say that chunk of memory held a password -- you'd definitely want to clear it after use, even if you immediately free it and never plan to read it again.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#66

Post by EdiX » Sat, Oct 24, 2015, 6:21 PM UTC

copsarebastards wrote: post #62

EdiX wrote: post #50

Earlier quoted context omitted.

Yes, the C language specification is a bit shit, it leaves too much leeway to compilers so that C compiler for broken, niche architectures can be written. However, I disagree with you. All this badness in the C specification did not stop people from writing reasonable C compilers for reasonable architectures for decades. The real problem here is competition. Gcc is in a competition with clang to produce fast code whi…

> This is a case of following the letter of the law (in this case the C standard) while disregarding its spirit: all the undefined behaviour was so that C compilers could accomodate for odd architectures while remaining close to the metal, not so that compiler programmers could go out of their way to turn their compiler into a mine field. Computers don't have spirits; they work as you tell them to work, to the letter…

>Computers don't have spirits

But standards committee do.

>Rust is extremely impressive because they've found so many ways to do high-level programming while maintaining low-level performance. But they can only do that because they have the benefit of the 4 decades of programming language research that have occurred since the basics of C were designed.

I doubt rust could be ported to a 8bit PIC microcontroller, or to a 6502 keeping reasonable performance characteristics or letting the programmer take advantage of the platform quirks. It's not just "4 decades of programming language research" it's also that it's intended to work only on "modern" processors.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#67

Post by lmm » Sat, Oct 24, 2015, 6:21 PM UTC

copsarebastards wrote: post #62

EdiX wrote: post #50

Earlier quoted context omitted.

Yes, the C language specification is a bit shit, it leaves too much leeway to compilers so that C compiler for broken, niche architectures can be written. However, I disagree with you. All this badness in the C specification did not stop people from writing reasonable C compilers for reasonable architectures for decades. The real problem here is competition. Gcc is in a competition with clang to produce fast code whi…

> This is a case of following the letter of the law (in this case the C standard) while disregarding its spirit: all the undefined behaviour was so that C compilers could accomodate for odd architectures while remaining close to the metal, not so that compiler programmers could go out of their way to turn their compiler into a mine field. Computers don't have spirits; they work as you tell them to work, to the letter…

You don't need 4 decades of programming language research to specify that e.g. signed integer overflow either returns an implementation-defined value or the program aborting. There are languages older than C that allowed the useful forms of bit-twiddling but offered much stronger safety guarantees.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#68

Post by angersock » Sat, Oct 24, 2015, 6:24 PM UTC

copsarebastards wrote: post #57

manawy wrote: post #51

Earlier quoted context omitted.

Yes, I'm really happy that GCC optimize that code away. Most of us don't care about security issue too much when using C/C++. We do use it for performance, and use it mostly locally. GCC is a very versatile code. It's ok that it makes secure code difficult to write because it's not what most of us is doing. Not being completely secure is ok, not being optimized is not.

Yeah, I'd go so far as to argue that if you need security, you probably shouldn't be writing C.

What you and a lot of other people are missing, cheerful in your use of other languages, is that your runtimes and native extensions usually depend on insecure C code.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#69

Post by lmm » Sat, Oct 24, 2015, 6:25 PM UTC

copsarebastards wrote: post #28

the_mitsuhiko wrote: post #20

Inknow people are quick to complain about programmers relying on UB here but this really is a long standing disagreement with the gcc folk. They are language lawyers of the worst sort and do not consider security implications being a point of discussion :(

> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. --Charles Babbage The modern version of this seems to be: "Mr. Babbage, I put the…

I'm reminded of http://mjg59.livejournal.com/108257.html :

"POSIX says this is fine, so any application that expected this behaviour is already broken by definition. But this is rules lawyering. POSIX says that many things that are not useful are fine, but doesn't exist for the pleasure of sadistic OS implementors. POSIX exists to allow application writers to write useful applications. If you interpret POSIX in such a way that gains you some benefit but shafts a large number of application writers then people are going to be reluctant to use your code. You're no longer a general purpose filesystem - you're a filesystem that's only suitable for people who write code with the expectation that their OS developers are actively trying to fuck them over."

GCC doesn't, or at least shouldn't, exist to implement the ANSI standard. It exists to help people to write useful programs.

Re: “gcc will quietly break nearly half of all the packages that it compiles”

#70

Post by copsarebastards » Sat, Oct 24, 2015, 6:25 PM UTC

makomk wrote: post #56

EdiX wrote: post #50

Earlier quoted context omitted.

Yes, the C language specification is a bit shit, it leaves too much leeway to compilers so that C compiler for broken, niche architectures can be written. However, I disagree with you. All this badness in the C specification did not stop people from writing reasonable C compilers for reasonable architectures for decades. The real problem here is competition. Gcc is in a competition with clang to produce fast code whi…

More specifically, they're competing to produce the fastest code for software that follows the C specification to the letter. That's not necessarily the same as producing the fastest code that actually achieves the intended goal. For example, if I recall correctly the popular Opus codec overflows signed integers when decoding invalid data, and so long as this can be guaranteed to produce some (possibly implementation…

> More specifically, they're competing to produce the fastest code for software that follows the C specification to the letter. That's not necessarily the same as producing the fastest code that actually achieves the intended goal.

That's true, but "producing the fasted code that actually achieves the intended goal" is not a problem which can be solved by a compiler--the compiler can't read your mind.

Attempting to represent a reasonable approximation of reading your mind is the responsibility of the specification. You can definitely do better than C, but I doubt you could have done better 4 decades ago when C was designed.