Undefined Behavior in C and C++ (2024) • Hacker News Board

Re: Undefined Behavior in C and C++ (2024)

#221

Post by vlovich123 » Wed, Aug 13, 2025, 3:00 PM UTC

ncruces wrote: post #211

vlovich123 wrote: post #191

Earlier quoted context omitted.

If a newer version of that value is also stored in a register and not yet flushed to memory, should the compiler know to insert that flush for your or is reading a stale value ok? For what it’s worth there’s a reason you’re supposed to do this kind of access through memcpy, not by dereferencing made up pointers. > There exists an instruction to do that load in Wasm, there's a builtin to check that 12345 points to add…

> If a newer version of that value is also stored in a register and not yet flushed to memory, should the compiler know to insert that flush for your or is reading a stale value ok? Any value would be OK. There are aliasing rules to follow, and it's OK if those crater performance when you start using integer-to-pointer conversions a lot. Is that a problem? But in this instance, assume I don't even care. > For what it…

> What good does this bring us if it means clang can't be used to target platforms where direct memory access is a thing?

I don’t know about this specific instance, but generally disallowing aliasing enables huge performance gains. That’s something Rust does to a great amount and one of the reasons real world code bases are faster in C than C++ (the other being a much better standard library with better defaults for containers and whatnot)

Re: Undefined Behavior in C and C++ (2024)

#222

Post by JonChesterfield » Wed, Aug 13, 2025, 6:35 PM UTC

jcranmer wrote: post #220

JonChesterfield wrote: post #212

Earlier quoted context omitted.

This is rather woven in with type themed alias analysis which makes a hard distinction tricky. E.g realloc doesn't work under either, but the provenance issue probably only shows up under no-strict-aliasing. I like pointer tagging because I like dynamic language implementations. That tends to look like "summon a pointer from arithmetic", which will have unknown to the compiler provenance, which is where the deref wit…

I think you're misunderstanding pointer provenance, and you're being angry at a model that doesn't exist. The failure mode of pointer provenance is converting an integer to a pointer to an object that was never converted to an integer. Tricks like packing integers into unknown bits or packing pointers into floating-point NaNs don't violate pointer provenance--it's really no different from passing a pointer to an exte…

That's definitely possible. The UB if no provenance information is available belief comes from https://www.cl.cam.ac.uk/~pes20/cerberus/clarifying-provenan..., in particular

> access via a pointer value with empty provenance is undefined behaviour

I'm annoyed that casting an aligned array of bytes to a pointer to a network packet type is forbidden, and that a pointer to float can't be cast to a pointer to a simd vector of float, and that malloc cant be written in C, but perhaps those aren't provenance either.

Re: Undefined Behavior in C and C++ (2024)

#223

Post by uecker » Wed, Aug 13, 2025, 7:01 PM UTC

tialaramex wrote: post #219

uecker wrote: post #214

Earlier quoted context omitted.

I do not see how this adds a substantial cost and it is required for C programs to work correctly and the C standard carefully describes the exact situation where unspecified values are chosen - so the idea that the compiler is then free to break this is clearly in contradiction to the wording. Clang got this wrong and I assume mostly fixed it, because non-frozen values caused a lot inconsistency and other bugs.

Like I said, maybe I'm not understanding which "unspecified values" we're talking about. The freeze semantic is a problem when we've said only that we don't know what value is present (typically one or more mapped but unwritten bytes) and so since we never wrote to this RAM the underpinning machine feels confident to just change what is there. Which means saying "No" isn't on the compiler per se. The OS and machine (…

No, but jemalloc uses a kernel API that has the behavior and IMHO is is then non-conforming (when using this API, which I think is configurable). The Facebook bug should be taken as a clear sign that this behavior is a terrible idea and not something to be even blessed by modifying the standard. When the original kernel API was introduced, it was already pointed out that the behavior is not ideal. There is no fundamental reason (including performance reasons) this has to behave in this way. It is just bad engineering.

Re: Undefined Behavior in C and C++ (2024)

#224

Post by jcranmer » Wed, Aug 13, 2025, 7:19 PM UTC

JonChesterfield wrote: post #222

jcranmer wrote: post #220

Earlier quoted context omitted.

I think you're misunderstanding pointer provenance, and you're being angry at a model that doesn't exist. The failure mode of pointer provenance is converting an integer to a pointer to an object that was never converted to an integer. Tricks like packing integers into unknown bits or packing pointers into floating-point NaNs don't violate pointer provenance--it's really no different from passing a pointer to an exte…

That's definitely possible. The UB if no provenance information is available belief comes from https://www.cl.cam.ac.uk/~pes20/cerberus/clarifying-provenan... , in particular > access via a pointer value with empty provenance is undefined behaviour I'm annoyed that casting an aligned array of bytes to a pointer to a network packet type is forbidden, and that a pointer to float can't be cast to a pointer to a simd vec…

> The UB if no provenance information is available belief comes from https://www.cl.cam.ac.uk/~pes20/cerberus/clarifying-provenan..., in particular

That's an old document. In particular, it's largely arguing for a PVI provenance model (i.e., integers carry provenance information), whereas the current TS is relying on a PNVI provenance model (i.e., integers do not carry provenance information). https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2577.pdf is the last draft pre-TS-ification (i.e., has all the background information to understand it).

> I'm annoyed that casting an aligned array of bytes to a pointer to a network packet type is forbidden, and that a pointer to float can't be cast to a pointer to a simd vector of float, and that malloc cant be written in C, but perhaps those aren't provenance either.

That's all strict aliasing rules, not pointer provenance rules. (Well, malloc has issues with living in the penumbra of the C object model). The big thing that provenance prevents you from doing is writing memcpy in C (since char access of a pointer counts as exposing the pointer, whereas the PNVI model makes memcpy a non-exposing operation).

Re: Undefined Behavior in C and C++ (2024)

#225

Post by tialaramex » Wed, Aug 13, 2025, 8:27 PM UTC

uecker wrote: post #223

tialaramex wrote: post #219

Earlier quoted context omitted.

Like I said, maybe I'm not understanding which "unspecified values" we're talking about. The freeze semantic is a problem when we've said only that we don't know what value is present (typically one or more mapped but unwritten bytes) and so since we never wrote to this RAM the underpinning machine feels confident to just change what is there. Which means saying "No" isn't on the compiler per se. The OS and machine (…

No, but jemalloc uses a kernel API that has the behavior and IMHO is is then non-conforming (when using this API, which I think is configurable). The Facebook bug should be taken as a clear sign that this behavior is a terrible idea and not something to be even blessed by modifying the standard. When the original kernel API was introduced, it was already pointed out that the behavior is not ideal. There is no fundame…

But far from "The compiler shouldn't allow this" what we're talking about here is platform behaviour. My impression is that virtual machines often just do this, so it may be that even your OS has no idea either.

Re: Undefined Behavior in C and C++ (2024)

#226

Post by quietbritishjim » Wed, Aug 13, 2025, 11:03 PM UTC

kazinator wrote: post #195

quietbritishjim wrote: post #194

Earlier quoted context omitted.

> The "project" is all the files going into a program supplied other than by the implementation. Most of my most recent comment is addressing the possibility that you meant this. As I said, there is no such concept to the compiler. It isn't passed any list of files that could be included with #includr, only the .c files actually being compiled, and the directories containing includable files. The fact that your IDE s…

There is no "compiler", "IDE" or "include path" in the wording of the ISO C standard. A set of files is somehow presented to the implementation in a way that is not specified. Needless to say, a file that is included like "globals.h" but is not the base file of a translation unit will not be indicated to the implementation as the base of a translation unit. Nevertheless it has to be somehow present, if it is required…

I only discussed things like include directories and IDEs, which are not part of the standard, because I am trying in good faith to understand how you could have come to your position. There is nothing in the standard like the "set of files is somehow presented to the implementation" (in a sense that includes header files) so I reasoned that maybe you were thinking of something outside the standard.

Instead, the standard says that the include directive:

> searches a sequence of implementation-defined places for a header ... and causes the replacement of that directive by the entire contents of the header.

(Note that it talks about simply substituting in text, not anything more magical, but that's digressing.)

It's careful to say "places" rather than "directories" to avoid the requirement that there's an actual file system, but the idea is the same. You don't pass the implementation every individual file that might need to be included, you pass in the places that hold them and a way to search them with a name.

Maybe you were confused by that part of the standard you quoted in an earlier comment.

One part of that says "The text of the program is kept in units called source files, (or preprocessing files) in this document." But the "source files" aren't really relevant to the include directive – those are the top-level files being compiled (what you've called "base files").

The next sentence you quoted says "A source file together with all the headers and source files included via the preprocessing directive #include is known as a preprocessing translation unit." But "all the headers" here is just referring to files that have been found by the search mechanism referred to above, not some explicit list.

Re: Undefined Behavior in C and C++ (2024)

#227

Post by ncruces » Thu, Aug 14, 2025, 1:34 AM UTC

uecker wrote: post #215

ncruces wrote: post #183

Earlier quoted context omitted.

Why can't it work though? And I'm taking about both things. Integer arithmetic that produces pointers that are just out of bounds of an object. Why can't this work? Why can't the compiler assume that, since I explicitly converted a pointer to an integer, the pointed-to object can't be put into a register, or made to go out or scope early? Second, fabricating pointers. If I have a pointer to mmap/sbrk memory, shouldn'…

Integer arithmetic working correctly, converting a pointer to an integer, and converting an integer back to the same object is something which should work. This is what we made sure to guarantee in the provenance TS. Making this work for memory outside of any object or converting back to a different object whose address was not converted to an integer previously is difficult. It can be made to work, but then you need…

> Note also that historically, the C standard left things UB exactly so that implementations can use it for extensions (e.g. mmap). The idea that UB == invalid program is fairly recent misinformation, but we have to react to it and make things at least implementation-defined (which also meant a bit something else before).

I'll just finish by saying: yes please. And thank you for bearing with me.

Re: Undefined Behavior in C and C++ (2024)

#228

Post by kazinator » Thu, Aug 14, 2025, 3:53 AM UTC

quietbritishjim wrote: post #226

kazinator wrote: post #195

Earlier quoted context omitted.

There is no "compiler", "IDE" or "include path" in the wording of the ISO C standard. A set of files is somehow presented to the implementation in a way that is not specified. Needless to say, a file that is included like "globals.h" but is not the base file of a translation unit will not be indicated to the implementation as the base of a translation unit. Nevertheless it has to be somehow present, if it is required…

I only discussed things like include directories and IDEs, which are not part of the standard, because I am trying in good faith to understand how you could have come to your position. There is nothing in the standard like the "set of files is somehow presented to the implementation" (in a sense that includes header files) so I reasoned that maybe you were thinking of something outside the standard. Instead, the stan…

My position doesn't revolve around the mechanics of preprocessing. Say we have somehow given the implementation a translation unit which has #include . Say we did not give the implementation a file winkle.h; we did not place such a file in any of the places where it searches for include files.

OK, now suppose that the implementation resolves #include and replaces it with tokens.

The subsequent processing is what my position is concerned with.

My position is that since the standard doesn't define what those tokens are, the behavior is not defined.

In other words, a conforming implementation can respond #include with any behavior whatsoever.

- It can diagnose it as not being found.

- It can replace it with the token __wipe_current_directory which that same implementation then understands as a compile-time instruction to wipe the current direcctory.

- Or any other possibility at all.

This all has to do with the fact that the header is not required to exist, but may exist, and if it does exit, it may have any contents whatsoever, including non-portable contents which that implementation understands which do weird things.

It is not required to document any of it, but if it does, that constitutes a documented extension.

A conforming implementation can support a program like this:

  #include 

  program HelloWorld;
  begin
    WriteLn('Hello, World!');
  end.

All that has to do is regurgitate a token like __pascal_mode. This token is procesed by translation phase 7, which tells the implementation to start parsing Pascal, as an extension.

Re: Undefined Behavior in C and C++ (2024)

#229

Post by eru » Thu, Aug 14, 2025, 3:57 AM UTC

m-schuetz wrote: post #201

eru wrote: post #197

Earlier quoted context omitted.

How is this obnoxious? C and C++ have their strengths, but rapid prototyping is generally not seen to be amongst them. This shouldn't be any more controversial than saying that pure Python is generally slow.

They are pretty much the best choice for prototyping 3D apps and GPU algorithms. They're fast, powerful, and don't impose restrictions - you can do whatever and however. It also helps that CUDA is C++.

> They're fast, powerful, and don't impose restrictions [...]

By that metric assembly is the best prototyping language.

Re: Undefined Behavior in C and C++ (2024)

#230

Post by rurban » Thu, Aug 14, 2025, 6:46 AM UTC

gpderetta wrote: post #88

ncruces wrote: post #83

Earlier quoted context omitted.

Why? What specific optimization do you have in mind that prevents me from doing an aligned 16/32/64-byte vector load that covers the address pointed to by a valid char*?

Casting a char pointer to a vector pointer and doing vector loads doesn't violate provenance, although it might violate TBAA. Regarding provenance, consider this: void bar(); int foo() { int * ptr = malloc(sizeof(int)); *ptr = 10; bar(); int result = *ptr; free(ptr); return result; } If the compiler can track the lifetime of the dynamically allocated int, it can remove the allocation and covert this function to simpl…

Then show me the compiler which tells the user that it removed this dead code. There is even an assignment removed, which violates all expectations