Live data from Hacker News

Undefined Behavior in 2017

blog.regehr.org

111–120 of 120 posts

Re: Undefined Behavior in 2017

#111
post #107
post #97

Earlier quoted context omitted.

The performance culture isn't just C. Far from it. Rust was susceptible to Stack Clash because they ripped out their existing stack probing to gain a few percent increase in performance. The article for this thread literally says, "For many use cases ASan is the better choice because it has much less overhead." IME Valgrind does a _much_ better job of detecting memory issues, partly because ASan only detects issues d…

> The performance culture isn't just C. Far from it Sure, but it is where it is more visible, specially micro-optimizing code as it is being written, without validating if it really matters to the application's use case with a profiler. The school of systems programming languages from Algol side, was that correctness was much more relevant than pure performance, Algol dialects for systems programming already had Rust…

I've never met a C programmer or C compiler author who felt correctness should take a backseat to performance. And as I demonstrated, the Rust designers are hardly immune to poor decision making on this score.

Someone else corrected me regarding the stack probes--Rust didn't have probes but actually checked the size of the stack. It's also worth pointing out that often times decisions are premised on maintenance burdens. Performance was one excuse for removing the stack checking in Rust; the other was the maintenance burden of maintaining a feature outside of LLVM.

That goes directly to the heart of the undefinedness issue in C. Doing the "obviously correct" thing wrt to choosing between possible undefined- or implementation-defined behaviors isn't as easy as we think. Good compilers are built on abstractions and code reuse, like any other piece of well-written software. Doing the correct thing in some obvious cases can result in having to make non-obvious tradeoffs elsewhere, possibly having to choose between adding (or maintaining) complexity, or removing a powerful feature, such as deterministic stack overflow prevention.

The history of C (indeed, Unix and the whole "worse is better" school of design) is about favoring less complexity in the tooling while shifting more of the burden onto the user. That's a judgment not about runtime performance, but a very unintuitive judgment about the performance and efficiency of programmer time; that you quickly reach a point of diminishing returns, much sooner than most people believe, putting complexity into, and relying on intrinsic features of, the tooling; that it's often better to make it easier to implement and integrate specialized tools, and easier to reuse existing tools as part of a more focused solution. From that perspective the design of C makes a ton more sense, even if reasonable people can still disagree about the results.

The notion that C--the language design, the ecosystem--is principally about performance is something mostly held by inexperienced programmers. It's a strawman of critics. There were plenty of other languages from the 1970s that were capable of generating faster code than C. Certainly performance was and continues to be of significant concern, but C is hardly unique in that sense. If the debate about how to balance performance with other issues stands-out, I'd argue that's primarily because of C's ubiquitous position and how it and the computing industry (including hardware industry) has co-evolved.

When something seems so "obvious" (security vs performance), yet everybody continues to seemingly repeat the same dumb mistakes, often times what we're missing is that our premises are flawed. Maybe C didn't become ubiquitous because of it's performance, but more so because it was easy to port and easy to build upon. After several years Rust still has a single implementation. Even Go has two production-quality implementations. That's not a criticism of Rust, just an observation that the "security vs performance" axis is hardly the only relevant dimension for judgment and explanation.

Re: Undefined Behavior in 2017

#112

Earlier quoted context omitted.

Not to belabor the point, but... > the only bullet-proof way to limit the scope of unsafe code is at the module boundary with privacy I understand how this is meant in the context of that page, and it is true that modules protect against users messing with modules' internal invariants. But does that also work the other way around? In the example in https://gankro.github.io/blah/only-in-rust/#unbound-lifetime... it's…

I would argue that this is not the caller making the mistake; it's this function. That is, since this function is safe, any safe code should be able to call it and not generate UB. It's not the caller's fault here, it's the incorrect implementation.

> It's not the caller's fault here, it's the incorrect implementation.

I agree. But then this shows that it's possible to write "safe" Rust code that only calls "safe" external code and still (to a first approximation) have the possibility of undefined behavior showing up at any point. In other words, Rust's famous static, compiler-enforced guarantees are not guarantees at all, more like firm promises.

Nothing wrong with that; it's easy to write memory-corrupting code in other safe-by-default languages like Haskell or OCaml as well. But it seems like Rust's marketing materials do try to suggest otherwise, and many people get wrong impressions (look at the first post in this thread, and the other comments on this article saying that Rust's "safe" code is 100% free from undefined behavior).

Re: Undefined Behavior in 2017

#113
post #5
post #3

Earlier quoted context omitted.

None exist in (safe) Rust.

Sometimes you hit undefined behavior in LLVM when compiling safe Rust, but that is considered a bug in the compiler. https://github.com/rust-lang/rust/issues/10184

By the same logic, you could do something in C that isn't considered undefined behavior but acts in an undefined way when compiled with Clang

Re: Undefined Behavior in 2017

#114
post #76
post #50

Earlier quoted context omitted.

It doesn't matter if the language defines the behaviour of, say, offsetting an array beyond its declared bounds. That doesn't change the general undecidability of statically proving the offset will always be within the bounds at compile-time - in other words, you need runtime checks in some cases (it's just a matter of whether the compiler or the programmer supplies them).

Nah, with some help from the programmer, you can produce those proofs. It's not too hard. The halting problem is impossible to solve for arbitrarily evil programs, but that doesn't mean that it's impossible to write programs that are easy to analyse.

Here are two nice blog posts on the state of the art of this "not too hard" problem: https://maniagnosis.crsr.net/2017/06/AFL-brute-force-search.... https://maniagnosis.crsr.net/2017/06/AFL-bug-in-quicksearch....

Possible? Yes. Too time-consuming and fiddly for the economic realities of 99.9% of all programmers? Yes, that too.

Re: Undefined Behavior in 2017

#115

Earlier quoted context omitted.

I would argue that this is not the caller making the mistake; it's this function. That is, since this function is safe, any safe code should be able to call it and not generate UB. It's not the caller's fault here, it's the incorrect implementation.

> It's not the caller's fault here, it's the incorrect implementation. I agree. But then this shows that it's possible to write "safe" Rust code that only calls "safe" external code and still (to a first approximation) have the possibility of undefined behavior showing up at any point. In other words, Rust's famous static, compiler-enforced guarantees are not guarantees at all, more like firm promises. Nothing wrong…

I mean, safe code is. Again, it's the unsafe that's at fault here.

Your point about other languages is exactly what I was going to say; unsafe is like an FFI layer. Nobody says that Ruby isn't memory safe because it can call into C code, and if someone messes up the C, well, it's at fault. "It's memory safe except for FFI" is a mouthful, and so people generally let the exceptions slide. Same with Rust.

Re: Undefined Behavior in 2017

#116
post #76

Earlier quoted context omitted.

Nah, with some help from the programmer, you can produce those proofs. It's not too hard. The halting problem is impossible to solve for arbitrarily evil programs, but that doesn't mean that it's impossible to write programs that are easy to analyse.

Here are two nice blog posts on the state of the art of this "not too hard" problem: https://maniagnosis.crsr.net/2017/06/AFL-brute-force-search.... https://maniagnosis.crsr.net/2017/06/AFL-bug-in-quicksearch.... Possible? Yes. Too time-consuming and fiddly for the economic realities of 99.9% of all programmers? Yes, that too.

The economic realities would be quite different if software companies were legally accountable for their defects, just like in many other industries.

Re: Undefined Behavior in 2017

#117
post #80
post #67

Earlier quoted context omitted.

Well, I don't know about half of these programming languages. However: - ADA requires garbage collection - Modula-2 and Object pascal have manual memory management, i.e.: "new" and "delete" - together with double-free, use-after-free, and dangling pointers... I agree that C is by far worse, but please recognize that at least some of the undefined behavior problems are really hard to solve. EDIT: Oh, and we haven't st…

What about learning to understand English? "writing mostly safe systems programs." Do you understand what mostly means? Ada and Modula-2 have multi-threading as part of their ISO/ANSI language standard. Also you don't seem to know much about Ada given your GC remark, but here is some learning. https://archive.fosdem.org/2016/schedule/event/ada_memory/ Of course there are still use cases where Algol derived languages…

> What about learning to understand English?

That's uncalled for, man...

> Of course there are still use cases where Algol derived languages are still unsafe, however those use cases are a tiny portion of what happens in C land.

On that I agree - there are many problems we wouldn't have if we were using, say, pascal or modula.

> Also you don't seem to know much about Ada given your GC remark, but here is some learning.

I'm sorry, I don't have time to watch that. However, I know that ADA has either GC or manual memory management (with the deallocation method explicitly marked as unsafe). Thus, as far as i know, constructing "use-after-free" in Ada is possible - which is undefined behavior (and that was my point).

Re: Undefined Behavior in 2017

#118
post #18
post #15

Earlier quoted context omitted.

They are called undefined behaviors because the compiler developers are free to do whatever they want when they come across a code that instigates an undefined behavior. Mostly new compilers now a days do handle such behavior with a compiler error or warning. So if you try to do an out of bound shift a new compiler might throw a warning, but according to the C standard it does not have to. In fact instead of throwing…

> So if you try to do an out of bound shift a new compiler might throw a warning, but according to the C standard it does not have to. Right, for doing so needs to determine that this is happening, which neither GCC and Clang can do for fairly trivial cases like for (i = 0; i

Yes they don't, I assumed they would, but they dont. I just wanted to emphasize the fact that they don't have to. Perhaps an unbound shift was a wrong example.. thanks for catching that...

Re: Undefined Behavior in 2017

#119
post #76

Earlier quoted context omitted.

Nah, with some help from the programmer, you can produce those proofs. It's not too hard. The halting problem is impossible to solve for arbitrarily evil programs, but that doesn't mean that it's impossible to write programs that are easy to analyse.

I think that statically guaranteeing something like in-bound indexing or absence of division by zero you need depended types, which is not exactly easy nor mainstream. And even with dependent types, many non-evil, otherwise correct programs will be rejected.

Yes, for the general problem you want dependent types (or similar).

There are middle grounds, of course: you can let the compiler figure out on its own what it can, and let it insert runtime checks for the other cases.

And add some way for the programmer to declare that eg "warn me if this can't be optimized" or "trust me, this is safe" or even "here's a proof for you".

But in any case, dependent types are possible---undecidability is not a hard barrier here.

Re: Undefined Behavior in 2017

#120
post #116

Earlier quoted context omitted.

Here are two nice blog posts on the state of the art of this "not too hard" problem: https://maniagnosis.crsr.net/2017/06/AFL-brute-force-search.... https://maniagnosis.crsr.net/2017/06/AFL-bug-in-quicksearch.... Possible? Yes. Too time-consuming and fiddly for the economic realities of 99.9% of all programmers? Yes, that too.

The economic realities would be quite different if software companies were legally accountable for their defects, just like in many other industries.

That would mostly mean that only big companies that can afford a herd of lawyers would be able to produce software.

At least that how bank regulation works out in practice.

Post reply on HN