Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clang emits memcpy for std::swap, which can introduce undefined behavior

llvm.org

Post reply on HN
Print view View original

21–30 of 83 posts

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#21

Post by dkopi »

hendzen wrote: post #3

Specifically in the case of a self-swap, which is a somewhat odd thing to do in the first place.

Imagine reversing a string:

  while(pStart
If swap doesn't work for a self swap, this code would break in the case of an odd length string. It's an easy fix in this case, but being able to swap an element with itself is convenient.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#22

Post by yelnatz »

nikic wrote: post #13

The bug report contains a quote from the C++ standard: "An rvalue or lvalue t is swappable if and only if t is swappable with any rvalue or lvalue, respectively, of type T" What does this statement mean and why is it not tautological?

If you extract it for only one case, it's easier to read.

  >An rvalue t is swappable if and only if t is swappable with any rvalue of type T

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#23

Post by ktRolster »

boulos wrote: post #6
majewsky wrote: post #4

Earlier quoted context omitted.

The second one sounds like big fun when you LD_PRELOAD a different memcpy (say, for debugging purposes) and spend hours and hours trying to figure out why it is not being called.

Yes, it's a nightmare, particularly on OS X with its funny loader semantics that make replacing malloc and free correctly very painful. For example, you can easily LD_PRELOAD your way out of malloc() but did you remember that asprintf also calls an allocation routine and won't use your new malloc? Enjoy.

For debugging purposes, it can sometimes be easier to include a header file "mem_debug.h" or whatever, and redefine malloc:

  >#define malloc(a) debug_malloc(a, __FILE__, __LINE__)
  >#define free(a)   debug_free(a, __FILE__, __LINE__)
  >.
  >.
  >.
Then in your debug_malloc() function you can track where every allocation took place. Not always applicable but nice when it works.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#24

Post by Matheus28 »

dkopi wrote: post #21
hendzen wrote: post #3

Specifically in the case of a self-swap, which is a somewhat odd thing to do in the first place.

Imagine reversing a string: while(pStart If swap doesn't work for a self swap, this code would break in the case of an odd length string. It's an easy fix in this case, but being able to swap an element with itself is convenient.

A more common case, would be swapping a vector element with the back of the vector and popping it off, which is kinda common.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#26

Post by slededit »

mehrdada wrote: post #20

The compiler causing undefined behavior on its own reminds me of one of the fun bugs we found[1] with 32-bit x86 clang where one compiler pass fools another into thinking there's an undefined behavior by combining three struct field initializations into a single 4-byte load, but the struct itself was 3-bytes long. Another pass looked at this and thought there's an undefined behavior writing past the struct boundary,…

Compilers are way too overzealous eliminating code because of undefined behavior. Legally it is allowed and I'm sure it helps in synthetic benchmarks. But writers of real programs won't be impressed of the perf gains when you stop running half their program.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#27

Post by kazinator »

The compiler is the implementation; it cannot cause "undefined behavior".

What it can be is "nonconforming".

A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming.

An implementation is not bound by the standard in its own use of the standard library functions. The implementation can use a function like memcpy in ways that would be formally undefined if they occured in user code; its implementation of memcpy just has to harmonize with that use, so that the intended behavior is ensured. This is because implementations can add their own requirements to areas that the standard leaves undefined. For instance, an implementation can add the requirement to its memcpy that the copy may overlap, if the destination has a lower address. Then, the implementation can generate code which uses memcpy that way, or make internal uses of memcpy from other library functions which use it that way. It's just using its own (possibly not publicly documented) extension.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#28

Post by electrograv »

haberman wrote: post #10

Given how aggressively compilers (including Clang) are exploiting undefined behavior these days, it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.

> it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.

I have suddenly lost a great deal of respect and confidence for LLVM/Clang after reading this bug thread; I value software reliability above all else, and being able to write reliable software without the compiler negating that effort. It seems I can no longer be assured this is even a priority of my compiler any more, with Clang. All the more reason to switch to Rust I suppose.

That the Clang people are trying to deflect this bug as "fine, no fix necessary" is kind of disgusting software engineering behavior. Compilers spontaneously introducing undefined behavior in another layer (to the point of causing codegen warnings from perfectly correct C code, is as clearly cut a BUG as any, independent of current API implementations outside of the ISO spec.

If you want to make undefined API calls the norm, then make a new API where the behavior you want is defined. Invoking undefined behavior of a decoupled library that users can swap out is flat out indefensible.

That they are reluctant to fix this makes me shy away from ever using Clang/LLVM, which is sad because before I had always held them with very high respect.

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#29

Post by comex »

boulos wrote: post #6
majewsky wrote: post #4

Earlier quoted context omitted.

The second one sounds like big fun when you LD_PRELOAD a different memcpy (say, for debugging purposes) and spend hours and hours trying to figure out why it is not being called.

Yes, it's a nightmare, particularly on OS X with its funny loader semantics that make replacing malloc and free correctly very painful. For example, you can easily LD_PRELOAD your way out of malloc() but did you remember that asprintf also calls an allocation routine and won't use your new malloc? Enjoy.

Hmm, why won't it? Maybe in the past when asprintf and malloc were located in the same dylib, since OS X dylibs generate direct calls to other public functions in the same image (rather than going through the PLT as GNU does by default), but for many years /usr/lib/libSystem.B.dylib has been split into several reexported libraries in /usr/lib/system, so there shouldn't be an issue there. Probably better to use this technique though (should still work in the latest version but needs mprotect):

http://eatmyrandom.blogspot.com/2010/03/mallocfree-intercept...

Re: Clang emits memcpy for std::swap, which can introduce undefined behavior

#30

Post by Dylan16807 »

kazinator wrote: post #27

The compiler is the implementation; it cannot cause "undefined behavior". What it can be is "nonconforming". A correct program which hits that situation continues to have well-defined behavior (which we know from the text of the program and the standard). Just the implementation isn't handling the requirements correctly; it is not conforming. An implementation is not bound by the standard in its own use of the standa…

The compiler has different layers. Layer X is nonconforming, which causes undefined behavior in layer M.

Layer M's memcpy doesn't have to have the same semantics as the C standard's memcpy, but the bug report implies that it does.

Post reply on HN