Why do we need for an Undefined Behavior Annex to C++ • Hacker News Board

Re: Why do we need for an Undefined Behavior Annex to C++

#31

Post by GrumpySloth » Fri, Mar 01, 2024, 7:24 PM UTC

sgift wrote: post #20

jeffbee wrote: post #12

Earlier quoted context omitted.

This is my point precisely. Having proven to itself that 0 <= x < 8, the compiler could go on to remove any test for x < 8, right?

Yes, and the compiler does that. That's one of the reasons that iterators are preferred over for-loops with manual access in Rust. While compilers can for very simple cases determine that you will always stay within the bounds (as in your example, where it can find at compile time that the length will always be 8 and no access outside happens) with iterators the compiler doesn't need to know the actual length and can…

Pedantic note: compiler doesn’t elide bounds checks in code using iterators. They’re not there to begin with. Iterators in the standard library are implemented using unsafe code, which calls non-bounds-checking variants of functions getting elements from collections. There is nothing for the compiler left to do here regarding bounds-checking.

Re: Why do we need for an Undefined Behavior Annex to C++

#32

Post by deathanatos » Fri, Mar 01, 2024, 7:29 PM UTC

vlovich123 wrote: post #7

mike_hock wrote: post #6

Earlier quoted context omitted.

No, because UB is not avoidable in principle unless you create a walled garden/sandbox that must be enforced at non-zero runtime cost.

I don’t know. Safe rust has 0 UB although if I recall correctly things are still a bit messy with integer overflow. I could imagine a world where we define annotations around “signed integer will never overflow” that you can add to hit paths but otherwise disallow optimizing around that UB and for all other UB require an explicit annotation acknowledging it or just a warning and a missed optimization instead. The cur…

Safe Rust has 0 UB, but the person you're responding to qualified that with:

> that must be enforced at non-zero runtime cost.

Some of Rust's behavior that permits this is not zero runtime costs. Bounds-checking, for example, has a non-zero cost: the bounds check!

Now, I greatly prefer Rust's approach: I'd rather have the marginal CPU cost: I value my time far higher, at least until the profiler speaks up, and in many cases the optimizer is pretty good at eliminating checks that I myself might look at and go "but I know $condition is true here!" — so too does the optimizer. (And these days, Godbolt makes testing that very simple.)

And in the worse case, there's unsafe{}, and it is at least explicitly labelled as such to the next reader.

> things are still a bit messy with integer overflow

Integer overflow is well-defined behavior, but the behavior depends on compile settings. (https://doc.rust-lang.org/book/ch03-02-data-types.html#integ...) I do hope that someday, the debug behavior becomes the behavior: IME most overflows are errors / the author did not intend for it to occur. And the "panic on overflow" behavior is removable by simply using one of the functions that specifies an overflow behavior, in which case then the author's intention is just explicitly stated.)

Re: Why do we need for an Undefined Behavior Annex to C++

#33

Post by bluGill » Fri, Mar 01, 2024, 7:38 PM UTC

marcosdumay wrote: post #16

mike_hock wrote: post #5

Earlier quoted context omitted.

Once it's complete, make it authoritative. Any mention of undefined behavior should be required to reference the corresponding item in the UB annex.

And even better, start reducing the list after it's complete, by pushing things like integer overflow into implementation defined; several uses of inc/decrement operators into compilation error; and etc.

I don't want integer overflow to be implementation defined though, I want it undefined so my compiler can optimize my code for the fact that I don't overflow my integers in the first place.

Re: Why do we need for an Undefined Behavior Annex to C++

#34

Post by sgift » Fri, Mar 01, 2024, 7:45 PM UTC

GrumpySloth wrote: post #31

sgift wrote: post #20

Earlier quoted context omitted.

Yes, and the compiler does that. That's one of the reasons that iterators are preferred over for-loops with manual access in Rust. While compilers can for very simple cases determine that you will always stay within the bounds (as in your example, where it can find at compile time that the length will always be 8 and no access outside happens) with iterators the compiler doesn't need to know the actual length and can…

Pedantic note: compiler doesn’t elide bounds checks in code using iterators. They’re not there to begin with. Iterators in the standard library are implemented using unsafe code, which calls non-bounds-checking variants of functions getting elements from collections. There is nothing for the compiler left to do here regarding bounds-checking.

Pedantic, but interesting - didn't know that. Thanks!

Re: Why do we need for an Undefined Behavior Annex to C++

#35

Post by fweimer » Fri, Mar 01, 2024, 8:29 PM UTC

Kranar wrote: post #18

jpcfl wrote: post #17

Earlier quoted context omitted.

> Safe rust has 0 UB Safe Rust aims for 0 UB, but I don't think you can make the claim that it absolutely has no UB. This program SEGFAULTs on my system (macOS), because it's reading an invalid memory address due to a stack overflow: const N: usize = 1024*1024*1024; fn main() { let var: [u8; N] = [0; N]; println!("var: {:?}", var); }

Safe Rust has no undefined behavior. Undefined behavior does not mean no crashing, it means that the semantics of the program are undefined. Rust's semantics are to abort on a stack overflow. A language like C or C++ have no such semantics, they may abort or they may continue running and producing jibberish.

How does Rust implement this on targets where LLVM does not implement stack clash protection?

Re: Why do we need for an Undefined Behavior Annex to C++

#36

Post by jpcfl » Fri, Mar 01, 2024, 8:51 PM UTC

Kranar wrote: post #18

jpcfl wrote: post #17

Earlier quoted context omitted.

> Safe rust has 0 UB Safe Rust aims for 0 UB, but I don't think you can make the claim that it absolutely has no UB. This program SEGFAULTs on my system (macOS), because it's reading an invalid memory address due to a stack overflow: const N: usize = 1024*1024*1024; fn main() { let var: [u8; N] = [0; N]; println!("var: {:?}", var); }

Safe Rust has no undefined behavior. Undefined behavior does not mean no crashing, it means that the semantics of the program are undefined. Rust's semantics are to abort on a stack overflow. A language like C or C++ have no such semantics, they may abort or they may continue running and producing jibberish.

The fact that this program results in reading/writing an unmapped memory address means it’s doing an out-of-bounds access. It segfaults on macOS because the runtime/OS has allocated the stack such that the overflow results in a bad memory access, but that is a behavior of the runtime/OS/hardware, not the language.

I guarantee I could exploit this on a system that does not have virtual memory, or a runtime that does not have unmapped addresses at the end of the stack, to, say, manipulate the contents of another thread’s stack. Therefore, this behavior is undefined.

Re: Why do we need for an Undefined Behavior Annex to C++

#37

Post by avgcorrection » Fri, Mar 01, 2024, 9:28 PM UTC

jpcfl wrote: post #36

Kranar wrote: post #18

Earlier quoted context omitted.

Safe Rust has no undefined behavior. Undefined behavior does not mean no crashing, it means that the semantics of the program are undefined. Rust's semantics are to abort on a stack overflow. A language like C or C++ have no such semantics, they may abort or they may continue running and producing jibberish.

The fact that this program results in reading/writing an unmapped memory address means it’s doing an out-of-bounds access. It segfaults on macOS because the runtime/OS has allocated the stack such that the overflow results in a bad memory access, but that is a behavior of the runtime/OS/hardware, not the language. I guarantee I could exploit this on a system that does not have virtual memory, or a runtime that does n…

Report it.

Re: Why do we need for an Undefined Behavior Annex to C++

#38

Post by dzaima » Fri, Mar 01, 2024, 9:48 PM UTC

jpcfl wrote: post #36

Kranar wrote: post #18

Earlier quoted context omitted.

Safe Rust has no undefined behavior. Undefined behavior does not mean no crashing, it means that the semantics of the program are undefined. Rust's semantics are to abort on a stack overflow. A language like C or C++ have no such semantics, they may abort or they may continue running and producing jibberish.

The fact that this program results in reading/writing an unmapped memory address means it’s doing an out-of-bounds access. It segfaults on macOS because the runtime/OS has allocated the stack such that the overflow results in a bad memory access, but that is a behavior of the runtime/OS/hardware, not the language. I guarantee I could exploit this on a system that does not have virtual memory, or a runtime that does n…

The language runtime can require that the OS & hardware always results in an exception on stack overflow (or, alternatively, compile in explicit checks for it). You running the program in an environment without that is, technically, just as wrong as running it on a system where integer addition does multiplication.

Now perhaps this means that there are real rust deployments that are "wrong", but that shouldn't include regular sane standard systems, and embedded users should know the tradeoffs.

https://godbolt.org/z/Y75KTT87M:

    .LBB3_1:
            sub     rsp, 4096
            mov     qword ptr [rsp], 0
            cmp     rsp, r11
            jne     .LBB3_1

That's a loop at the start of your 'main' that probes the stack specifically to ensure a segfault definitely happens if your array didn't fit on the stack.

Re: Why do we need for an Undefined Behavior Annex to C++

#39

Post by vlovich123 » Fri, Mar 01, 2024, 10:01 PM UTC

Kranar wrote: post #30

vlovich123 wrote: post #25

Earlier quoted context omitted.

I don't know if it's technically UB or well defined. The crash is a SEGFAULT and not a panic/abort, but it's probably a SEGFAULT due to guard pages. Still, it's possible to evade guard pages so if you access var[X] such that X points to the heap, it's possible you're reading aliased memory which would be UB in safe Rust. EDIT: Going to take it back. I'm unable to create a situation where I create a large stack array…

Panics are not quite the same as an abort in Rust. Most notably a panic can be caught and execution can resume so as to gracefully terminate the application, but an abort is an immediate termination, a go to jail do not pass go kind of situation. An out of bounds access in Rust will result in a panic but a stack overflow is an abort.

A segfault would imply it's not an abort either although it seems like it has been converted to a proper abort in newer versions of Rust.

Re: Why do we need for an Undefined Behavior Annex to C++

#40

Post by quelsolaar » Fri, Mar 01, 2024, 10:12 PM UTC

wahern wrote: post #26

Notably, the C standard does have an undefined behavior annex, Annex J.2, since C99. It's not mentioned in the post, and this omission in tandem with repeated usage of "C and C++" at various points might suggest otherwise to some people.

And C has a UB study group and we are working on a technical rapport that in great detail describes how UB works in C, and a comprehensive list of sample code for all known UB.