Live data from Hacker News

Clang emits memcpy for std::swap, which can introduce undefined behavior

llvm.org

31–40 of 83 posts

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#31

The compiler causing undefined behavior on its own reminds me of one of the fun bugs we found[1] with 32-bit x86 clang where one compiler pass fools another into thinking there's an undefined behavior by combining three struct field initializations into a single 4-byte load, but the struct itself was 3-bytes long. Another pass looked at this and thought there's an undefined behavior writing past the struct boundary,…

Compilers are way too overzealous eliminating code because of undefined behavior. Legally it is allowed and I'm sure it helps in synthetic benchmarks. But writers of real programs won't be impressed of the perf gains when you stop running half their program.

Rather, they are getting zealous in eliminating code because of the assumption that there is no undefined behavior. The scope of what undefined behaviors do this is increasing and that breaks programs that make some assumptions on traditional compiler behavior, or that have hidden bugs.

I believe that C should have safer semantics, plus standard-defined modes of operation which restore the unsafe semantics selectively.

The optimizer then must obey whichever semantics in effect (safer or less safe).

Then, in addition to telling your compiler you want a certain optimization level, you could tell it which standard-defined modes of operation you would like.

Currently, this is already de facto the case outside of the standard. For instance, users of GCC who want to do certain pointer aliasing without unpredictable behavior tell the compiler to be in a mode in which that is allowed (effectively an altered C dialect) using -fno-strict-aliasing.

This kind of thing should be a standard, portable feature. Look, everywhere in this program, I want nice left-to-right evaluation within all expressions and among initializers and function arguments. Except in super-fast.c; when compiling super-fast.c to super-fast.o, please do it in a mode where unspecified-evaluation-order semantics applies and optimize accordingly. (I might even want this on a more finer-grained scale than translation units.)

We need safety, and we need to sometimes throw away safe semantics in exchange for better speed (whereby we still ensure that the code we are writing is correct with regard to the weakened semantics; i.e. we promise to the compiler that even if it relaxes the evaluation order, or whatever, we know what we are doing and everything is cool).

It would be good to do this without compiler-specific guesswork.

Optimization control alone over a language with a single set of semantics (which is largely unsafe) is a bad way to achieve this.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#32

The compiler is the implementation; it cannot cause "undefined behavior". What it can be is "nonconforming". A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming. An implementation is not bound by the standard in its own use of the standa…

Are you saying that Clang guarantees its own memcpy implementation will be used? I don't know; but it seems odd that if this is fully intended that the compiler/linker would generate warnings from its own handling of perfectly valid C code (as shown in the bug).

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#33

The compiler is the implementation; it cannot cause "undefined behavior". What it can be is "nonconforming". A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming. An implementation is not bound by the standard in its own use of the standa…

The compiler has different layers. Layer X is nonconforming, which causes undefined behavior in layer M. Layer M's memcpy doesn't have to have the same semantics as the C standard's memcpy, but the bug report implies that it does.

Yes, if "undefined behavior" doesn't just mean "a situation where the ISO language standard doesn't impose a requirement", but has a broader meaning like "behavior not defined by anyone at all".

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#34

Given how aggressively compilers (including Clang) are exploiting undefined behavior these days, it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.

Hi, LLVM and Clang developer here.

I don't see anyone being blasé about this. The issue comes down to something really simple:

- The compiler can emit substantially simpler code if it has a guarantee from the platform's implementation of memcpy.

- The compiler authors thought they had such a guarantee and so chose to leverage it.

Now, maybe they don't have that guarantee on all platforms. If that's the case, its flat out and simply a bug in the compiler. And you'll find folks on that bug arguing that it doesn't actually hold, which is good and important.

But it's actually quite reasonable to have a guarantee from a platform's libc that this will work and leverage that. The compiler isn't generating C code or code bound by any standard. This is about two parts of the implementation of C agreeing about what internal invariants will hold.

The problem this bug highlights (and it is a real problem, and not one anyone takes lightly) is that we have insufficient agreement about these invariants, both for certain platforms (glibc) and even within LLVM. Fixing this remains absolutely critical, but may only involve documenting the agreement.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#35

Given how aggressively compilers (including Clang) are exploiting undefined behavior these days, it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.

> it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves. I have suddenly lost a great deal of respect and confidence for LLVM/Clang after reading this bug thread; I value software reliability above all else, and being able to write reliable software without the compiler negating that effort. It seems I can no longer be assured this is even a pri…

Understand that some of the folks who don't think this is a bug own one of the libc implementations being used. So for their platform, it really is 100% correct and valid.

The folks working on other platforms seem universally unhappy with that bug.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#36

Earlier quoted context omitted.

The compiler has different layers. Layer X is nonconforming, which causes undefined behavior in layer M. Layer M's memcpy doesn't have to have the same semantics as the C standard's memcpy, but the bug report implies that it does.

Yes, if "undefined behavior" doesn't just mean "a situation where the ISO language standard doesn't impose a requirement", but has a broader meaning like "behavior not defined by anyone at all".

The compiler is using the ISO language standard to define the behavior of an internal piece. It is then using that piece in the wrong way. It's not generic "not defined" behavior, it is specific C "undefined behavior".

Think of that part of the transformed code as still C for relevant purposes.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#37

Earlier quoted context omitted.

> it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves. I have suddenly lost a great deal of respect and confidence for LLVM/Clang after reading this bug thread; I value software reliability above all else, and being able to write reliable software without the compiler negating that effort. It seems I can no longer be assured this is even a pri…

Understand that some of the folks who don't think this is a bug own one of the libc implementations being used . So for their platform, it really is 100% correct and valid. The folks working on other platforms seem universally unhappy with that bug.

If it's not 100% valid on 100% platforms and 100% of all future Libc revisions (which is impossible if it relies on undefined behavior and can link with different Libc versions), then it's not 100% correct and valid.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#38

The compiler is the implementation; it cannot cause "undefined behavior". What it can be is "nonconforming". A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming. An implementation is not bound by the standard in its own use of the standa…

Are you saying that Clang guarantees its own memcpy implementation will be used? I don't know; but it seems odd that if this is fully intended that the compiler/linker would generate warnings from its own handling of perfectly valid C code (as shown in the bug).

No, Clang cannot guarantee its own memcpy implementation will be used. But it can be aware of which implementation will be used and make specific assumptions about that implementation in some cases.

The specific cases are when it has a genuine guarantee provided by that implementation vendor that it is OK. In the case discussed on this bug, at least one implementation provides that guarantee (Darwin). The problem is that not all provide it, and further than other parts of LLVM have started assuming that the memcpy more closely matches C semantics. At least something will have to change here, but the C semantics are really relevant at all, or what is "valid C code".

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#39

Earlier quoted context omitted.

Are you saying that Clang guarantees its own memcpy implementation will be used? I don't know; but it seems odd that if this is fully intended that the compiler/linker would generate warnings from its own handling of perfectly valid C code (as shown in the bug).

No, Clang cannot guarantee its own memcpy implementation will be used. But it can be aware of which implementation will be used and make specific assumptions about that implementation in some cases . The specific cases are when it has a genuine guarantee provided by that implementation vendor that it is OK. In the case discussed on this bug, at least one implementation provides that guarantee (Darwin). The problem is…

Clang is not a complete implementation without a library, right? Whoever puts Clang together with a library becomes an integrator who provides a language implementation. It's up to that integrator to ensure that the result conforms.

Clang could be also used "freestanding". Then the user might be informed: please supply your own memcpy function with such and such characteristics, since generated code needs it.

If those characteristics are just those described in ISO C, but the compiler needs something more, then that's a problem between clang and the user.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#40

Earlier quoted context omitted.

Understand that some of the folks who don't think this is a bug own one of the libc implementations being used . So for their platform, it really is 100% correct and valid. The folks working on other platforms seem universally unhappy with that bug.

If it's not 100% valid on 100% platforms and 100% of all future Libc revisions (which is impossible if it relies on undefined behavior and can link with different Libc versions), then it's not 100% correct and valid.

On at least one platform, all future libc revisions will abide by this requirement.

And neither Clang nor LLVM are generating platform agnostic code. The code only has meaning for the specific platform being targeted.

Post reply on HN