Live data from Hacker News

I Do Not Know C: Short quiz on undefined behavior (2015)

kukuruku.co

171–180 of 189 posts

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#171
post #81
post #76

Earlier quoted context omitted.

It allows the optimizer to assume away the halting problem; all nontrivial loops are obligated to halt.

I read this as, will confuse the snot out of hapless newbie programmers trying to learn C via stepping though their code in an IDE. While providing no practical benefit to programmers writing production code.

It is necessary in order for the compiler to do transformation optimizations which do impact production code. Newbies shouldn't be writing production code without guidance anyways IMO.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#172
post #170

Earlier quoted context omitted.

In the referenced case the introduced error involved a reference to a null pointer but there was still no exploitable security hole. The exploit was enabled when the compiler removed an explicit check. The null dereference was an error, but it was not a security issue on its own.

Why didn't the kernel panic when it tried to access NULL?

Why is a compiler writer attempting to legislate the kernel address space?

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#173

Earlier quoted context omitted.

Actually that's not correct. The standard says the behavior is up to the compiler. The compiler author took that as a license to produce a non truth preserving transformation of the code. The actual current clang behavior also satisfies the standard as written.

> The standard says the behavior is up to the compiler. I think this statement is correct, but it's the kind of thing people say when they confuse implementation defined behavior and undefined behavior. And that distinction is key. Implementation defined behavior means the compiler gets to choose what it will do, document the choice, and then stick to it. Undefined behavior means that the program is invalid, but the…

So the standard does not require the compiler to e.g. remove the "redundant" check for null or assume that signed integers don't cycle values on overflow, but _permits_ the compiler to do so. Thus, we have two problems: a poorly thought out standard which permits dangerous compiler behavior and poorly thought out compiler that jumps at the chance.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#174

Earlier quoted context omitted.

> The standard says the behavior is up to the compiler. I think this statement is correct, but it's the kind of thing people say when they confuse implementation defined behavior and undefined behavior. And that distinction is key. Implementation defined behavior means the compiler gets to choose what it will do, document the choice, and then stick to it. Undefined behavior means that the program is invalid, but the…

So the standard does not require the compiler to e.g. remove the "redundant" check for null or assume that signed integers don't cycle values on overflow, but _permits_ the compiler to do so. Thus, we have two problems: a poorly thought out standard which permits dangerous compiler behavior and poorly thought out compiler that jumps at the chance.

I never said the standard required the compiler to remove the redundant check; just that it is allowed to, and I gave an example of why it might.

But since the compiler is allowed to, programmers have to act accordingly.

*

Yes, this does lead to a situation similar to what C.A.R. Hoare described:

"Now let me tell you about yet another overambitious language project. ... I was a member and even chairman of the Technical Committee No. 10 of the European Computer Manufacturers Association. We were charged ... with ... the standardization of a language to end all languages. ... I had studied with interest and amazement, even a touch of amusement, the four initial documents describing a language called NPL. ... Each was more ambitious and absurd than the last in its wishful speculations. Then the language began to be implemented and a new series of documents began to appear at six-monthly intervals, each describing the final frozen version of the language, under its final frozen name PL/1.

"But to me, each revision of the document simply showed how far the initial Flevel implementation had progressed. Those parts of the language that were not yet implemented were still described in free-flowing flowery prose giving promise of unalloyed delight. In the parts that had been implemented, the flowers had withered; they were choked by an undergrowth of explanatory footnotes, placing arbitrary and unpleasant restrictions on the use of each feature and loading upon a programmer the responsibility for controlling the complex and unexpected side-effects and interaction effects with all the other features of the language" ( http://zoo.cs.yale.edu/classes/cs422/2014/bib/hoare81emperor... , pg. 10).

In The Design and Evolution of C++, Stroustrup mentioned that occasionally when designing a feature, he could think of multiple possibilities: often one would involve convoluted rules that the compiler could enforce, and the other would be a simple rule (or set of rules) that the compiler couldn't necessarily enforce. He said he generally chose the simple rules, even if he didn't know how to have the compiler detect violations. So C++ ended up with things like the One Definition Rule (and violation is undefined behavior). I've never seen any similar statements from Ritchie or Thompson, but I suspect they followed a similar approach. Of course, today both languages are governed by committees, so how they balance trade-offs may have changed.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#175
post #170

Earlier quoted context omitted.

Why didn't the kernel panic when it tried to access NULL?

Why is a compiler writer attempting to legislate the kernel address space?

Give me a break. The kernel developers know C. They know what "undefined behavior" means.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#176
post #70

Earlier quoted context omitted.

In the really simple cases (like accessing a float as a long, or similar), you're right. The problem is the interpretation that's been applied to aggregate types: struct a { int variant; }; struct b { int variant; long data; }; I have a pointer to a struct b - can I cast it to a pointer to struct a and still dereference 'variant'? It has the correct type and is guaranteed to live at the start of the struct in both ca…

> I have a pointer to a struct b - can I cast it to a pointer to struct a and still dereference 'variant'? I think it is a bit of a gray area, but personally I've always held the opinion/understanding of no, that is invalid. The C standard does make one point fairly clear on strict-aliasing, which is the idea that strict-aliasing revolves on the idea that an object can only be considered to be one type of data (Or a…

Kinda. It's actually surprising how close it comes to skirting this issue (And it does skirt it), but I believe it's actually possible to use BSD sockets without ever violating the strict-aliasing rule (Though of course, there are ways of using it which would arguably violate the rule). In most cases for BSD sockets, strict-aliasing is more likely going to be broken in the kernel, not your code.

Well, firstly it's pretty unsatisfying to hear that yes, this API contravenes strict aliasing restrictions, but only on the library side! - essentially that it is impossible to implement the sockets C API in C.

That aside, this still excludes long-standing examples like embedding a pointer to struct sockaddr in your client struct, which points to either a sockaddr_in, sockaddr_in6 or sockaddr_un depending on where that client connected from (well, you can still do it, but now you can't examine the sockaddr's sa_family member to see what type the address really is - you need to have a redundant, duplicate copy of that field in the client struct itself).

The situation is similar with sockaddr_storage. The whole point of that type is to allow you to stash either AF_INET or AF_INET6 addresses in the same object and then examine the ss_family field to see what it really is - the text in POSIX says:

  The  header shall define the sockaddr_storage structure, which shall be:

    Large enough to accommodate all supported protocol-specific address structures

    Aligned at an appropriate boundary so that pointers to it can be cast as pointers to protocol-specific address structures and used to access the fields of those structures without alignment problems
( http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys... )

Of course, it's also worth keeping in mind that the BSD sockets interface came before C89. You definitely wouldn't design it the same way if you were to do it today.

Well, the aforementioned sockaddr_storage came about after C89.

And wasn't C89 supposed to be about codifying existing practice, anyway?

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#177
post #176

Earlier quoted context omitted.

> I have a pointer to a struct b - can I cast it to a pointer to struct a and still dereference 'variant'? I think it is a bit of a gray area, but personally I've always held the opinion/understanding of no, that is invalid. The C standard does make one point fairly clear on strict-aliasing, which is the idea that strict-aliasing revolves on the idea that an object can only be considered to be one type of data (Or a…

Kinda. It's actually surprising how close it comes to skirting this issue (And it does skirt it), but I believe it's actually possible to use BSD sockets without ever violating the strict-aliasing rule (Though of course, there are ways of using it which would arguably violate the rule). In most cases for BSD sockets, strict-aliasing is more likely going to be broken in the kernel, not your code. Well, firstly it's pr…

> Well, firstly it's pretty unsatisfying to hear that yes, this API contravenes strict aliasing restrictions, but only on the library side! - essentially that it is impossible to implement the sockets C API in C.

Then it'd also be pretty unsatisfying to hear that it's impossible to write an OS kernel in standard C too - it requires compiler extensions for lots of various details. It's impossible to write a standard C library in nothing but standard C as well. I would absolutely agree, however, that the fact that you have to use an extension to get past strict-aliasing is unfortunate (It'd be nice if in a future C they add something like the `may_alias` extension into the standard). But you can do it, and OS code is definitely one where extensions are going to be rampant anyway. For example, the Linux Kernel disables strict-aliasing completely.

> That aside, this still excludes long-standing examples like embedding a pointer to struct sockaddr in your client struct, which points to either a sockaddr_in, sockaddr_in6 or sockaddr_un depending on where that client connected from (well, you can still do it, but now you can't examine the sockaddr's sa_family member to see what type the address really is - you need to have a redundant, duplicate copy of that field in the client struct itself).

Strictly speaking, that's not true. You can examine it directly, but it requires casting the `struct sockaddr * ` to a `sa_family_t * ` instead. This is valid because regardless of what type of object it really is, it must start with a `sa_family_t` entry, so you are treating it as the correct type (And by that notion, `sa_family_t` is allowed to alias any of the `struct sockaddr` types). Besides that, you could also use a `union` to combine all the possible `sockaddr`s that you're going to handle together with a `sockaddr_storage`. Then you can do things like normal and simply access the `sockaddr` through the correct union member.

> Well, the aforementioned sockaddr_storage came about after C89.

That it did. However, that was more about making the current API work with larger addresses (namely ipv6) not to fix the API. There really wasn't any other way to do it.

> And wasn't C89 supposed to be about codifying existing practice, anyway?

Some compilers implemented the strict-aliasing optimization (And did lots of other strange things), some did not. The C standards committee chose to go with strict-aliasing since it provides a lot of optimization opportunities.

All that said, I'm not saying things are perfect by any means, as our conversation here shows. I don't think things are nearly as bad as people tend to think, however. Generally speaking, unless you're doing something a little bit shady it's possible to avoid any strict-aliasing issues, and if it's really not possible there's generally a way to simply "turn it off", albeit that may result in a little less portable code - though if you're breaking strict-aliasing the portability of your code is already a bit suspect to begin with.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#178
post #176

Earlier quoted context omitted.

Kinda. It's actually surprising how close it comes to skirting this issue (And it does skirt it), but I believe it's actually possible to use BSD sockets without ever violating the strict-aliasing rule (Though of course, there are ways of using it which would arguably violate the rule). In most cases for BSD sockets, strict-aliasing is more likely going to be broken in the kernel, not your code. Well, firstly it's pr…

> Well, firstly it's pretty unsatisfying to hear that yes, this API contravenes strict aliasing restrictions, but only on the library side! - essentially that it is impossible to implement the sockets C API in C. Then it'd also be pretty unsatisfying to hear that it's impossible to write an OS kernel in standard C too - it requires compiler extensions for lots of various details. It's impossible to write a standard C…

Strictly speaking, that's not true. You can examine it directly, but it requires casting the `struct sockaddr * ` to a `sa_family_t * ` instead. This is valid because regardless of what type of object it really is, it must start with a `sa_family_t` entry,...

No, that isn't guaranteed by POSIX - it has to have an sa_family_t member, but it doesn't have to be the first one.

I also think it's problematic that the use explicitly contemplated by POSIX is considered ill-formed C.

Besides that, you could also use a `union` to combine all the possible `sockaddr`s that you're going to handle together with a `sockaddr_storage`. Then you can do things like normal and simply access the `sockaddr` through the correct union member.

I do not think this is that easy when you include sockaddr_un into the mix, because of the way that sockaddr doesn't include the full size of its path member. This is, in fact, the point that I throw up my hands and use -fno-strict-aliasing because the fact that pointer provenance, rather than just value and type, is important together with the fact that it's not actually clear whether you've correctly laundered the pointer through a union or not, makes it all too... grey.

Some compilers implemented the strict-aliasing optimization (And did lots of other strange things), some did not.

C compilers existed in 1989 that would assume different structure types with common initial sequences couldn't alias? With the "common initial sequence" carve-out in §3.3.2.3? I am sceptical...

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#179
post #124

Earlier quoted context omitted.

* No NULL pointer can be dereferenced. NULL pointers CAN be dereferenced, it all depends on the environment you run on.

In standard C NULL pointers cannot be dereferenced. Full stop, there is nothing to argue about it. There are environments that either lack memory protection and do not allow invalid pointer derereferences to be caught, which means that you can't rely on the MMU to catch mistakes. In this case either deal with silent errors or get a compiler that is able (at a cost) to sanitize any pointer access. There are other syst…

There are indeed such systems.

Embedded systems (which are frequently programmed in assembly or 'C' code).

Such systems very often map a small bit of high-speed (on chip) RAM to the first few bytes of address space. I very distinctly recall such an embedded system in a college course.

Re: I Do Not Know C: Short quiz on undefined behavior (2015)

#180
post #17

Earlier quoted context omitted.

I don't think #2 has been fully beaten to death yet. Assuming a platform where you don't segfault (say that 'page 0' variables are valid) and thus runtime does proceed; I still can't think of any /valid/ reason to eliminate the if that follows (focus line 2 in the comments). Under what set of logic does being able to de-reference a pointer confer that it's value is not 0 (which is what the test equates to)? In my opi…

C programmers expect dead code removal. Especially when the compiler also inlines functions (and, of course, inlining makes the biggest impact on short functions; and one way to get short functions is to have aggressive dead code removal). And macros can expand into very weird, but valid, code; so the statement that "nobody would ever write code like that" isn't relevant. The compiler may well have to handle unnatura…

OK, I think I see what SHOULD happen here.

It's been forever since I've actually compiled a C program not part of some package I was installing, BUT.

* There should be a warning flag, and it should be ON BY DEFAULT, that //all// code removal is stated, and the logic behind it given.

* It should be possible to elevate that condition to an error (and that too might even be the default).

Post reply on HN