Undefined behavior in C is a reading error • Hacker News Board

Re: Undefined behavior in C is a reading error

#481

Post by hvdijk » Sat, May 22, 2021, 2:10 PM UTC

aw1621107 wrote: post #471

hvdijk wrote: post #464

Earlier quoted context omitted.

Sure, but in the situation we were talking about, the user never wrote a definition for write(), and the user did not specify any library to include that provided a definition of write(). From the standard's perspective, that means there is no definition for it in the entire program. Keep in mind that the standard's perspective is somewhat different from how things work in practice. We know that on Unix-like systems,…

> and the user did not specify any library to include that provided a definition of write() Ah. I had assumed that that was implicit in "using write(2)", but seems that was a bad assumption. > there is also the concept of libraries, somewhat different from how the standard describes it In what way? You make an interesting point with the example. It's not something I had considered before. Would weak linkage (or a sim…

> In what way?

For the most part the standard does not address the existence of libraries other than the standard library, but 5.1.1.1 contains "Previously translated translation units may be preserved individually or in libraries." This, to me, suggests that from the standard's perspective, when you link in a library, you simply get that library, whereas on Unix systems, when you link in a static library, you specifically get those object files from the library needed to resolve not yet defined references, and when you link in a shared library, you get something where it becomes possible to have duplicate definitions where rules come into play as to which definition will end up used.

> You make an interesting point with the example. It's not something I had considered before. Would weak linkage (or a similar mechanism that allows for a provide-unless-the-user-already-did-so type of behavior) fall under an implementation extension, then?

Yes, I think so. Shared libraries implicitly have some sort of weak linkage already aside from the explicit weak linkage that you can get with e.g. GCC's __attribute__((weak)), but both forms count as extensions, I would say.

Re: Undefined behavior in C is a reading error

#482

Post by vyodaiken » Sat, May 22, 2021, 2:38 PM UTC

Rusky wrote: post #443

vyodaiken wrote: post #432

Earlier quoted context omitted.

What indicates the the current semantics won't change next week? After all, it is completely up to the compiler, no?

Are you serious? Again: it's not up to the compiler at all, it's up to the spec. The spec indicates the current semantics won't change next week. The semantics for non-UB code are completely fixed across optimization levels and compiler versions. Only code that invokes UB can break on these changes, and only because this code never had any specified semantics to begin with.

It is impossible to write C applications without invoking UB and your theory that UB behavior had no semantics is nonsensical. It may have no semantics that compilers currently feel they need to keep stable, but code that compiles and runs has semantics.

Re: Undefined behavior in C is a reading error

#483

Post by vyodaiken » Sat, May 22, 2021, 2:57 PM UTC

tom_mellior wrote: post #479

vyodaiken wrote: post #467

Earlier quoted context omitted.

I wanted to show that we don't need UB justified deletes to get good code generation. There was no need to break all that working code when we could have just told people that size_t counters worked better in loops on x86-64 than ints. A lot of C optimization could work that way - relying on cooperation between the compiler and programmers. Java can't do that because Java programmers rely on complex abstractions that…

> A lot of C optimization could work that way - relying on cooperation between the compiler and programmers. That is precisely how it works already . The reason your code has no bounds checks is exactly because the compiler can assume that you have done your part and ensured that all indices are in bounds. This is what "the compiler can ignore UB" is all about. The signed integer kerfuffle is just the same: The compi…

The problems with that argument are 1) The expectations changed with no notice. You can say it always was that way, but that's just not correct. The bounds check worked and then didn't, no matter what you think the standard "always said" (and the UB experts on the WG14 often find it impossible to say exactly what provisions mean, so claims that all this was ever clear are also wrong.) 2) deleting overflow check reduces the power of the language. The supposed work arounds are painful and have edge cases. 3) the example, and others, show that much UB "we assume it can't happen" "optimization" is unnecessary. You make the language more difficult to use, more prone to unpleasant surprise, and in return you provide an "optmization" that could easily be produced by other means. You're insisting on using a hammer as a fork and annoyed when people don't find it convenient.

Re: Undefined behavior in C is a reading error

#484

Post by vyodaiken » Sat, May 22, 2021, 2:58 PM UTC

I_Byte wrote: post #197

tomcam wrote: post #59

Earlier quoted context omitted.

> Vanilla C no longer gives them the tools to do that on a modern processor Can you elaborate on this point?

C was created during a time where instructions were executed linearly with no vectorization, memory was a flat space with no CPU caches, and there wasn’t a branch predictor that may or may not execute the correct program branch in advance. The list goes on but the rest is beyond my scope. C was designed for a now obsolete computer architecture model and over the years this old model has essentially become an abstract…

FYI https://www.yodaiken.com/2021/05/21/your-computer-is-a-fast-...

Re: Undefined behavior in C is a reading error

#485

Post by vyodaiken » Sat, May 22, 2021, 2:59 PM UTC

vyodaiken wrote: post #473

I_Byte wrote: post #465

Earlier quoted context omitted.

I was unaware. Thank you for the correction! Do you have any further reading on this topic?

I am sorry to say, I do not have a good short reference. There has to be one, though, I hope. The Hennesey Patterson books are the standard intros.

https://www.yodaiken.com/2021/05/21/your-computer-is-a-fast-...

Re: Undefined behavior in C is a reading error

#486

Post by vyodaiken » Sat, May 22, 2021, 4:07 PM UTC

optymizer wrote: post #428

vyodaiken wrote: post #225

Earlier quoted context omitted.

C is designed to allow programmers to insert or remove checks as performance tuning. Compiler UB "optimizations" that remove that ability from the programmer make the language unusable.

Not knowing what is UB in C is equivalent to not knowing a core part of the C language. In that situation, yes, C would seem unusable for people who have superficial knowledge of it.

Send Linux Torvalds a note and tell him he has only superficial knowledge of C.

Re: Undefined behavior in C is a reading error

#487

Post by cryptonector » Sat, May 22, 2021, 5:34 PM UTC

aw1621107 wrote: post #457

cryptonector wrote: post #415

Earlier quoted context omitted.

> ignoring the situation completely with unpredictable results That needn't mean "delete the code". TFA is certainly right that this situation is killing C. But then, so is C's old type model. Rust is clearly better, so it's just as well. That said, I think the C99 UB stance is a disaster for any language that might adopt it. Perhaps the standard should not define `1 (char )0`, or signed integer overflow. But it's ea…

> 1. write your own compiler that does it right (ETOOHARD now that clang didn't) The problem here is that "right" is not quite that black-and-white. To a lot of users, "right" is "my code appears to work," optionally with "at a given performance level." A new compiler that's slower, or changes semantics compared to some baseline, won't necessarily be seen as "better" unless there's some clear benefit, and "doesn't pe…

This is only a matter of opinion because this mistake was made at all in the spec. If it hadn't been you'd not be saying any of the above. Most UB in C has reasonable implementation. E.g., `sizeof(enum_type)` should be 4 if its value range permits it, else 8. E.g., 1-bit int bitfields should behave like 1-bit unsigned int bitfields. I already covered bit shifting, signed integer overflows, and NULL dereferences, which covers... most things. OK, there's aliasing, but that was a terrible mistake. Really, it's not that hard.

"Clang is slow" would not be a result. "Clang produces slow code" might be due to it not deleting "unpossible" code, but screw that, you can always go delete it yourself if it was dead. The compiler deleting such code is a terrible trap.

Re: Undefined behavior in C is a reading error

#488

Post by nullc » Sat, May 22, 2021, 10:27 PM UTC

vyodaiken wrote: post #454

nullc wrote: post #207

Earlier quoted context omitted.

> Strict aliasing is another category of UB that I'd consider gratuitous. Without it you cannot vectorize (or even internally re-order) many loops which are currently vectorizable because the compiler can't statically prove arguments won't alias otherwise.

That is completely false. Look up "restrict" and there are many other contexts. BTW, "prove" and "assume" are different things. This is an old argument, which in a just world would have been settled by Dennis Ritchie's comments.

I'm quite familiar with restrict, having added it to many codebases. It's a moderately fragile construct which in no way replaces the normal c memory model. (It's also significantly under exploited by most compilers, presumably since very little code is restrict annotated.)

Prove and assume are indeed different things, but what the compiler is able to do is _prove_ the validity of the transform within the context of the C abstract machine.

Plenty of sibling comments in this thread show concrete examples of significant optimizations lost with alias analysis (and/or non-wrapping signed integers) disabled.

If the compiler is not allowed to assume that the source code is valid then very few optimizations are safe at all. Clearly that isn't what you want, so I suppose that you would argue that being able to vectorize something like 90% of all currently vectorizable loops in existing code isn't worth the cost, and instead compilers should be able to assume that anything my alias anything unless the programmer has manually restrict annotated every variable that is written to in a loop?

I expect in that world you'd see lots of code run needlessly lots slower, and lots of other code needlessly get slathered over with restrict annotations as restrict turns into a reflex because leaving it out results in poor performance so regularly... resulting in incorrect annotations and the miscompilation you were hoping to avoid-- arguably the worst of all worlds.

Re: Undefined behavior in C is a reading error

#489

Post by Rusky » Sat, May 22, 2021, 11:17 PM UTC

vyodaiken wrote: post #482

Rusky wrote: post #443

Earlier quoted context omitted.

Are you serious? Again: it's not up to the compiler at all, it's up to the spec. The spec indicates the current semantics won't change next week. The semantics for non-UB code are completely fixed across optimization levels and compiler versions. Only code that invokes UB can break on these changes, and only because this code never had any specified semantics to begin with.

It is impossible to write C applications without invoking UB and your theory that UB behavior had no semantics is nonsensical. It may have no semantics that compilers currently feel they need to keep stable, but code that compiles and runs has semantics .

That's not what I (or the standard, or compiler writers, or programming language researchers) mean by "semantics."

From the perspective of defining and specifying a programming language, when we say "semantics" we mean the set of rules for an abstract machine, or a similar formalism. If those rules don't specify the result of an operation, it's like the machine gets stuck- like dividing by zero in a proof, there is no correct way to proceed. The behavior is undefined.

Nobody is disputing that compilers will produce something for code with undefined behavior. They're just saying that there is absolutely no useful way to rely on it, because nobody has agreed on or even decided what it should be (and always for some specific reason!)

If that makes it impossible to use the language, that's not UB's problem. It's the design of the language, and the quality of the tools that surround it. There are ways to increase your confidence that a C application never invokes UB, and they're getting better all the time. (There are also lots of new languages that try to solve this in various ways that C can't!)

Those are the solutions we have. "Just don't have UB in C" or "just make compilers more predictable" are not very effective by comparison.

Re: Undefined behavior in C is a reading error

#490

Post by Thorrez » Sun, May 23, 2021, 4:41 AM UTC

andreareina wrote: post #480

Thorrez wrote: post #478

Earlier quoted context omitted.

Good point that checking for overflow of a multiplication beforehand is hard. But I don't think checking after the fact would be much easier. So thus a C modification to remove the undefined behavior probably wouldn't be very useful still.

The thing is that it's necessary, and (x86, not sure about arm) asm tells you via the d register if not the overflow flag. So why doesn't the C standard arguably provide a misuse-resistant way of doing so? Instead every project having to reinvent it and get it wrong if they do the "obvious" (but wrong) thing.

What would the misuse-resistant way be? Would you want the multiplication operator (*) to return 2 values: the number and a bool telling you whether it overflowed?

Yeah, a standard library function that does this would be good. But many people would just use * instead of this function, and so the problem would partially remain.