Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything in C is undefined behavior

blog.habets.se

Post reply on HN
Print view View original

701–710 of 748 posts

Re: Everything in C is undefined behavior

#701

Post by HarHarVeryFunny »

casey2 wrote: post #641
nullpwr wrote: post #428

Earlier quoted context omitted.

> The language should not let you create an an invalid pointer, or at least warn you when you are doing so completely agree!

That's a nonsensical statement, a language cannot warn you, only a compiler can (-Wcast-align). The compiler can also decide what is and isn't an invalid pointer, this way the language avoids leaky abstractions.

It's obvious at the langauge level if you are creating, for example, an int pointer that is not guaranteed to be int-aligned. The only way to guarantee that an int pointer is int-aligned is if it comes from malloc* or new (C++), or if you take the address of an integer variable.

Any time you need to use a type cast to assign an int pointer, other than casting the output of malloc, then you potentially have a unaligned pointer, although only the compiler, knowing the alignment requirements of the target, would know if that is creating an invalid pointer or not.

* The C/C++ language (standard library) spec says that malloc must return memory that is aligned to meet the (target) requirements of all built in types, which is obviously a bit wasteful, as well not helpful for things like SIMD types that may be supported by libraries rather than built-in.

Re: Everything in C is undefined behavior

#702

Post by Xirdus »

jstimpfle wrote: post #635
Xirdus wrote: post #622

Earlier quoted context omitted.

In practice, GCC -O2 will happily erase entire swathes of code and turn perfectly logical source into nonsense assembly whenever it gets as much as a sniff of UB anywhere in the code path. Nobody would be talking about UB if GCC wasn't so aggressive in abusing the freedom UB gives. To paraphrase your earlier comment - you lose low-friction polymorphism (unpredictable compiler output causes a lot of friction). You los…

Then I'm almost ashamed to admit that I'm not sure I've ever witnessed any surprising form of UB in the wild. For example, I will reliably get segfaults on NULL dereference in practice. Typical manifestations of UB are entirely predictable and obvious. Of course I'm also running most code without most optimizations, most of the time, while developing. On the other hand, what I've observed with my own eyes is interest…

There was once a privilege escalation vulnerability in Linux kernel that only happened when compiled with optimizations. In kernel space, address 0 is just regular memory that can be read from and written to if there's a page mapped to it. But in C standard, reads and writes to null pointer are UB.

There was some function that read from a passed pointer unconditionally whether it's null or not. It made sense in context. Then it checked if the pointer is null - if it is do early return, if it's not do privileged operation. The pointer was null iff the user didn't have permissions to do the operation.

What GCC did is notice that a pointer is accessed before its null check. Since accessing a null pointer is UB, and GCC assumes UB never happens, it figured out the null check is superfluous. And removed the check and the early return. The pointer read stayed, mind you. The optimized function would unconditionally read from the pointer even if it's null, then unconditionally execute the privileged operation without checking permissions. That allowed obtaining root access from anywhere.

I saw a few other writeups of interesting UB behavior on The Old New Thing blog. I especially like the time travel one: https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63... (apologies to people of the future, links to MS devblogs tend to die often).

Re: Everything in C is undefined behavior

#703

Post by rssoconnor »

ncruces wrote: post #515
account42 wrote: post #189

Earlier quoted context omitted.

This is not a problem that C or C++ programmers actually encounter, ever.

I actually encountered it a couple weeks ago. Can you spot the infinite loop in this function? char* strcpy(char* restrict d, const char* restrict s) { stpcpy(d, s); return d; } I'll help. A call to `stpcpy` that ignores the return value can be swapped with a call to the (more likely to be optimized) `strcpy`. Since that's infinite recursion, and there is no forward progress, it's undefined behavior and anything goes…

Naming an externally linked function with the prefix "str" is by itself UB since that prefix is reserved for .

Re: Everything in C is undefined behavior

#704

Post by HarHarVeryFunny »

casey2 wrote: post #641
nullpwr wrote: post #428

Earlier quoted context omitted.

> The language should not let you create an an invalid pointer, or at least warn you when you are doing so completely agree!

That's a nonsensical statement, a language cannot warn you, only a compiler can (-Wcast-align). The compiler can also decide what is and isn't an invalid pointer, this way the language avoids leaky abstractions.

Different hardware targets obviously differ in many ways, not just alignment requirements, but things like endianness, integer representation, etc. C23 has finally said that signed integers must be in 2's complement representation, but this is only practically possible since all modern CPUs have also done that. Maybe endianness will be be mandated at some point since modern processors have also rallied around little endian ordering.

However, C has since day one chosen to define a leaky abstraction by putting efficiency above all else and having the language/implementation adapt to the hardware rather than vice versa (having the implementation hide hardware differences to implement some language-defined standard). The most obvious case of this is the size of the integer types short, int, and long, where the standard only says that "short C's unions could also be considered as providing a leaky abstraction, since it exposes differences between implementations - endianness, alignment/packing.

Then you get to things like size of address space, pointer sizes and representations, even amount of memory that a program may be able to allocate.

The only way a language could totally abstract away the underling hardware would be to base it on some lowest common denominator of hardware (or virtual machines) that it is willing to support, which would be totally impractical. The alternative is that you just accept the leaky abstraction and say that many things are implementation defined, not language defined.

Re: Everything in C is undefined behavior

#705

Post by Chinjut »

jstimpfle wrote: post #496
Chinjut wrote: post #464

Earlier quoted context omitted.

How is undefined behavior necessary for this transformation?

IIRC computation of the address is done by computing offset from base pointer as a multiplication in (32-bit) int , (like p + (i * sizeof (Foo)) . The right term might overflow, but due to signed overflow being UB, the compiler is able to assume that it does not, so the transformation to do the arithmetic entirely in (64-bit) pointer space is valid.

But surely the more natural approach than making this undefined behavior would be making the computation of a[i] take place in 64-bit pointer space rather than 32-bit int space? Why does the compiler need the freedom to emit nasal demons?

Re: Everything in C is undefined behavior

#706

Post by ncruces »

rssoconnor wrote: post #703
ncruces wrote: post #515

Earlier quoted context omitted.

I actually encountered it a couple weeks ago. Can you spot the infinite loop in this function? char* strcpy(char* restrict d, const char* restrict s) { stpcpy(d, s); return d; } I'll help. A call to `stpcpy` that ignores the return value can be swapped with a call to the (more likely to be optimized) `strcpy`. Since that's infinite recursion, and there is no forward progress, it's undefined behavior and anything goes…

Naming an externally linked function with the prefix "str" is by itself UB since that prefix is reserved for .

Right. I am doing a standard library replacement, and forgot I needed to compile freestanding. Oops. My bad.

So `str` and `mem` are reserved. But then so are `to` and `is` (by ). Just forget about having a function named `is_valid_user`.

And so are `mtx_`, `cnd_`, `thrd_`, `atomic_`, `memory_`...

Which is why... everything in C is UB.

Re: Everything in C is undefined behavior

#707

Post by SAI_Peregrinus »

jstimpfle wrote: post #658
steveklabnik wrote: post #647

Earlier quoted context omitted.

“Undefined behavior” is a term of art in programming languages that means something more specific than “the program may do something odd.” The compiler is not allowed to derive any assumptions from it. It only could if it were UB.

But did the rust compiler assume that the integer would not overflow? It did so in Debug mode where runtime checks were added. If it's not the case in Release mode, does that mean semantics are different between Debug and Release?

The semantics are well-defined in both modes. You can predict exactly what will happen in either case. In C, the semantics are not defined at all, you can't predict what will happen and it's allowed to change between compilations of the same source.

It will probably get omitted, since Undefined Behavior isn't allowed by the C abstract machine, but sadly compilers are allowed to emit code for UB in the source (partly because some UB is only detectable at runtime). Sometimes disabling optimizations will incorrectly allow codegen to run for source lines which have UB, tricking people into thinking that optimizations are breaking their program. Compilers are allowed to do this, since behaviors other than "omit the offending statement" are unfortunately allowed by the standard, so it's not a compiler bug.

Re: Everything in C is undefined behavior

#708

Post by steveklabnik »

jstimpfle wrote: post #658
steveklabnik wrote: post #647

Earlier quoted context omitted.

“Undefined behavior” is a term of art in programming languages that means something more specific than “the program may do something odd.” The compiler is not allowed to derive any assumptions from it. It only could if it were UB.

But did the rust compiler assume that the integer would not overflow? It did so in Debug mode where runtime checks were added. If it's not the case in Release mode, does that mean semantics are different between Debug and Release?

> But did the rust compiler assume that the integer would not overflow?

It did not.

> It did so in Debug mode where runtime checks were added.

It didn't assume in that case either. It did a well defined thing: add checks.

> If it's not the case in Release mode, does that mean semantics are different between Debug and Release?

Strictly speaking, the language doesn't know about "release mode", as that's a Cargo thing. But yes, in practice, the semantics are different based on various things: it could be debug vs release, it could also be flags that control the behavior. But that's still distinct from "undefined behavior" as a concept. The behavior is well defined, with multiple possible options for behaviors.

Re: Everything in C is undefined behavior

#709

Post by simonask »

jstimpfle wrote: post #687
simonask wrote: post #685

Earlier quoted context omitted.

If you're aware of that technique, why were you asking about use cases for unaligned loads?

I'm saying that string search algorithms are _not_ a legitimate use case for unaligned loads.

You didn’t really say that, but feel free to share any reasons you might have to think so.

I don’t see any reason why it wouldn’t be perfectly fine on recent hardware, where unaligned loads are just as fast, and the cache pressure is identical for a linear search algorithm.

Re: Everything in C is undefined behavior

#710

Post by chowells »

tardedmeme wrote: post #667
chowells wrote: post #532

Earlier quoted context omitted.

You seem to be operating under the assumption "undefined behavior" means "the compiler authors can decide what to do." That's not what it means. It means "any program that causes this behavior to be triggered is not a valid C program, the programmer knows this and did not submit an invalid program, and the programmer explicitly prevented this from happening elsewhere in ways automated analysis cannot detect. Proceed…

It means the C standard does not specify what the program does. Other documents may still specify what the program does. And the program definitely still does something, whether specified or not.

> And the program definitely still does something, whether specified or not.

No. It most definitely does not mean this. Go read the series this is part of: https://blog.llvm.org/2011/05/what-every-c-programmer-should...

It is absolutely critical that people programming in C understand what real compilers in the real world do.

Post reply on HN