Live data from Hacker News

Clang vs. Clang

blog.cr.yp.to

61–70 of 405 posts

Re: Clang vs. Clang

#61
post #45

It’s worth noting that, on Intel CPUs, neither clang nor anything else can possibly generate correct code, because correct code does not exist in user mode. https://www.intel.com/content/www/us/en/developer/articles/t... Look at DOITM in that document — it is simply impossible for a userspace crypto library to set the required bit.

Couldn't you syscall into the kernel to set the flag, then return back into usermode with it set?

So your compiler is supposed to emit a pair of syscalls each function that does integer math? Never mind that a pair of syscalls that do WRMSR may well take longer than whatever crypto operation is between them.

I have absolutely nothing good to say about Intel’s design here.

Re: Clang vs. Clang

#62

Earlier quoted context omitted.

Unfortunately GCC’s codegen for GCC’s x86 intrinsics headers is really remarkably awful at -O0, particularly around constant loads and broadcasts, because those usually use code that’s as naïve as possible and rely on compiler optimizations to actually turn it into a broadcast, immediate, or whatever. (I haven’t checked Clang.)

Clang tends to put everything on the stack at -O0 and actually try to do register allocation only as an optimization.

Generally? Sure, so does GCC, but that’s IME less impactful than a pessimized vectorized routine. (Corollary that I hit literally yesterday: exclusively at -O0, until you step past the opening brace—i.e. the function prologue—GDB will show stack garbage instead of the function arguments passed in registers.)

Re: Clang vs. Clang

#63

The author's Clang patch is interesting, but I wonder if what he really wants is, like, a new optimization level "-Obranchless" which is like O2/O3 but disables all optimizations which might introduce new conditional branches. Presumably optimizations that _remove_ branches are fine; it's just that you don't want any deliberately branchless subexpression being replaced with a branch. Basically like today's "-Og/-Odeb…

... except that even my idea fails to help with software math. If the programmer writes `uint128 a, b; ... a /= b` under -Obranchless, does that mean they don't want us calling a C++-runtime software division routine (__udiv3 or however it's spelled) that might contain branches? And if so, then what on earth do we do instead? — well, just give an error at compile time, I guess.

Yes, a compile failure would IMHO be the only useful result in that case.

Re: Clang vs. Clang

#64

I can't help but feel we're going to think of these as the bad old years, and that at some point we'll have migrated off of C to a language with much less UB. It's so easy to express things in C that compile but that the compiler couldn't possibly guess the intent of because C doesn't have a way to express it. For instance, in Python you can write something like: result = [something(value) for value in set_object] Be…

On the Python example, the downside is that, even though the order is unspecified, people may still rely on some properties, and have their code break when an optimizer changes the order. Basically the same as UB really, though potentially resulting in wrong results, not necessarily safety issues (at least not immediately; but wrong results can turn into safety issues later on). And, unlike with UB, having a "sanitizer" that verifies that your code works on all possible set orders is basically impossible.

gcc/clang do have a variety of things for providing low-level hints to the compiler that are frequently absent in other languages - __builtin_expect/__builtin_unpredictable, __builtin_unreachable/__builtin_assume, "#pragma clang loop vectorize(assume_safety)"/"#pragma GCC ivdep", more pragmas for disabling loop unrolling/vectorizing or choosing specific values. Biggest thing imo missing being some "optimization fences" to explicitly disallow the compiler to reason about a value from its source (__asm__ can, to an extent, do this, but has undesired side-effects, and needs platform-specific register kind names).

There's certainly potential in higher-level intent-based optimization though. Things coming to mind being reserving space in an arraylist before a loop with n pushes, merging hashmap lookups in code doing contains→get→put with the same key, simplifying away objects/allocations from ability to locally reason about global allocation behavior.

Re: Clang vs. Clang

#65

The author's Clang patch is interesting, but I wonder if what he really wants is, like, a new optimization level "-Obranchless" which is like O2/O3 but disables all optimizations which might introduce new conditional branches. Presumably optimizations that _remove_ branches are fine; it's just that you don't want any deliberately branchless subexpression being replaced with a branch. Basically like today's "-Og/-Odeb…

... except that even my idea fails to help with software math. If the programmer writes `uint128 a, b; ... a /= b` under -Obranchless, does that mean they don't want us calling a C++-runtime software division routine (__udiv3 or however it's spelled) that might contain branches? And if so, then what on earth do we do instead? — well, just give an error at compile time, I guess.

Not branchless, they just need it to be constant-time. That is definitely doable with pure software division.

Re: Clang vs. Clang

#66

> [..] whenever possible, compiler writers refuse to take responsibility for the bugs they introduced I have seldomly seen someone discredit their expertise that fast in a blog post. (Especially if you follow the link and realized it's just basic fundamental C stuff of UB not meaning it produces an "arbitrary" value.)

I think the author knows very well what UB is and means. But he’s thinking critically about the whole system. UB is meant to add value. It’s possible to write a language without it, so why do we have any UB at all? We do because of portability and because it gives flexibility to compilers writers. The post is all about whether this flexibility is worth it when compared with the difficulty of writing programs without…

The issue is that you’d have to come up with and agree on an alternative language specification without (or with less) UB. Having the compiler implementation be the specification is not a solution. And such a newly agreed specification would invariably either turn some previously conforming programs nonconforming, or reduce performance in relevant scenarios, or both.

That’s not to say that it wouldn’t be worth it, but given the multitude of compiler implementations and vendors, and the huge amount of existing code, it’s a difficult proposition.

What traditionally has been done, is either to define some “safe” subset of C verified by linters, or since you probably want to break some compatibility anyway, design a separate new language.

Re: Clang vs. Clang

#67

> [..] whenever possible, compiler writers refuse to take responsibility for the bugs they introduced I have seldomly seen someone discredit their expertise that fast in a blog post. (Especially if you follow the link and realized it's just basic fundamental C stuff of UB not meaning it produces an "arbitrary" value.)

I think the author knows very well what UB is and means. But he’s thinking critically about the whole system. UB is meant to add value. It’s possible to write a language without it, so why do we have any UB at all? We do because of portability and because it gives flexibility to compilers writers. The post is all about whether this flexibility is worth it when compared with the difficulty of writing programs without…

Even stipulating that part of the argument, the author then goes on a tear about optimizations breaking constant-time evaluation, which doesn’t have anything to do with UB.

The real argument seems to be that C compilers had it right when they really did embody C as portable assembly, and everything that’s made that mapping less predictable has been a regression.

Re: Clang vs. Clang

#68

> [..] whenever possible, compiler writers refuse to take responsibility for the bugs they introduced I have seldomly seen someone discredit their expertise that fast in a blog post. (Especially if you follow the link and realized it's just basic fundamental C stuff of UB not meaning it produces an "arbitrary" value.)

No, I think you're just speaking past each other here. You're using "bug" in reference to the source code. They're using "bug" in reference to the generated program. With UB it's often the case that the source code is buggy but the generated program is still correct. Later the compiler authors introduce a new optimization that generates a buggy program based on UB in the source code, and the finger-pointing starts. E…

> if your battery caught fire just because your CRUD app dereferenced NULL, nobody (well, nobody sane) would point the finger at the app author for forgetting to check for NULL.

I think pretty much anyone sane would and would be right to do so. Incorrect code is, well, incorrect and safety critical code shouldn’t use UB. Plus, it’s your duty as a software producer to use an appropriate toolchain and validate the application produced. You can’t offload the responsibility of your failure to do so to a third party (doesn’t stop people for trying all the time with either their toolchains or a library they use but that shouldn’t be tolerated and be pointed as the failure to properly test and validate it is).

I would be ashamed if fingers were pointed towards a compiler provider there unless said provider certified that its compiler wouldn’t do that and somehow lied (but even then, still a testing failure on the software producer part).

Re: Clang vs. Clang

#69
post #54

> compiler writers refuse to take responsibility for the bugs they introduced, even though the compiled code worked fine before the "optimizations". The excuse for not taking responsibility is that there are "language standards" saying that these bugs should be blamed on millions of programmers writing code that bumps into "undefined behavior" But that's not an excuse for having a bug; it's the exact evidence that it…

The problem is that c and c++ have a ridiculous amount of undefined behavior, and it is extremely difficult to avoid all of it. One of the advantages of rust is it confines any potential UB to unsafe blocks. But even in rust, which has defined behavior in a lot of places that are UB in c, if you venture into unsafe code, it is remarkable easy to accidentally run into subtle UB issues.

It’s true that UB is not intuitive at first, but “ridiculous amount” and “difficult to avoid” is overstating it. You have to have a proof-writing mindset when coding, but you do get sensitized to the pitfalls once you read up on what the language constructs actually guarantee (and don’t guarantee), and it’s not that much more difficult than, say, avoiding panics in Rust.

Re: Clang vs. Clang

#70

Earlier quoted context omitted.

No, I think you're just speaking past each other here. You're using "bug" in reference to the source code. They're using "bug" in reference to the generated program. With UB it's often the case that the source code is buggy but the generated program is still correct. Later the compiler authors introduce a new optimization that generates a buggy program based on UB in the source code, and the finger-pointing starts. E…

> if your battery caught fire just because your CRUD app dereferenced NULL, nobody (well, nobody sane) would point the finger at the app author for forgetting to check for NULL. I think pretty much anyone sane would and would be right to do so. Incorrect code is, well, incorrect and safety critical code shouldn’t use UB. Plus, it’s your duty as a software producer to use an appropriate toolchain and validate the appl…

> I think pretty much anyone sane would and would be right to do so. Incorrect code is, well, incorrect and safety critical code shouldn’t use UB

You missed the whole point of the example. I gave CRUD app as an example for a reason. We weren't talking safety-critical code like battery firmware here.

Post reply on HN