Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Being fair about memory safety and performance

thecodedmessage.com

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Being fair about memory safety and performance

#61

Post by pron »

pcwalton wrote: post #53
pron wrote: post #40

Earlier quoted context omitted.

Even if you look just at sound language guarantees, Zig is closer to Rust in terms of memory safety than to C or C++. That it doesn't make all of Rust's static guarantees doesn't put it in the same bucket as languages that make none. Once you reduce memory safety issues to below half of their current rate, there are different paths to achieving a good overall correctness story.

Zig is no way closer to Rust than C or C++ in this regard. Zig is in fact not appreciably safer than C or C++. All of the same classes of memory safety problems that are present in C or C++ are present in Zig. I am extremely doubtful of the claim that Zig eliminates 50% or more of memory safety problems.

As (safe) Zig eliminates overflows just as (safe) Rust does, and, like Rust, has not unsafe casts, I don't see how "all of the same classes of memory safety problems that are present in C or C++ are present in Zig." Also, Zig guarantees — just as Rust does — that all pointers and means of creating them are known, and precisely so, so analysis tools could work on Zig better than they do on C or C++. In C/C++, at best they could check pointers at the time of dereferencing, while for Zig they could do so at the time of deallocation. You once said that couldn't work because Zig would have dangling pointers lying around (presumably in addition to the one used for deallocation) because C does, but it might as well be the case that C programs leave dangling pointers because tools cannot detect them, which has affected the programming style, not the other way around.

It's OK to doubt claims/hopes about Zig, just as I doubt the claim that Rust can achieve more correct/secure programs than Zig for less effort. Without empirical data, there's really no way to know. So in the meantime, all there is to go on is personal appeal. But you cannot support your claim that Zig is "just like C" with the assumption that it is.

I think that, at the end of the day, there is what we might call an "ideological" difference between us. While we both accept that both language features and best practices — code reviews, tests, tools — reduce bugs, we draw the line of where it's worth to sacrifice one for the other in different places. It might also be the case that Rust's complexity doesn't sacrifice anything for you, but it does for me, as I'm uncomfortable with complex languages, and I think Rust easily makes the top four most complex "production" languages in history (together with C++, Ada, and Scala). So while even for me C is on the wrong side of my line (just as Idris is on the wrong side of yours), I reject extrapolating from C to Zig, because Zig makes many more guarantees at the language level, so without pertinent data about Zig, the question cannot be settled. If I thought Zig was "like C," I wouldn't have found it promising and so intriguing, either.

Having said all that, I don't want it to sound as if I'm willing to bet on Zig right now. I'm far too risk-averse for that. But I wouldn't bet on Rust right now, either. What it brings to the table doesn't offset, for me, its (still-)high risk. All I'm saying is that the revolutionary Zig hints at a promise of a new low-level language that could bring more to the table and justify the risk.

Re: Being fair about memory safety and performance

#62

Post by pron »

jbergens wrote: post #60
AnimalMuppet wrote: post #58

Earlier quoted context omitted.

> Expecting a language be better "enough" is an unrealistic one, you will be waiting forever. Disagree. I think the "market" (the set of programmers) defines "enough". When a language is enough better (in some area, doesn't have to be all areas) you see widespread adoption. C was enough better than PL/I, ALGOL, and assembly. Java was enough better than C++. (Why? Garbage collection, and the huge standard library.) So…

I neither a c++ programmer nor a rust one but I read things about programming. My take is that rust is near the breaking point where it becomes so commonly used that more teams and organizations will just start to use it. The risk of switching for at least parts or a code base will seem low and then it probably looks better enough. Of course some organizations and code bases are harder to change for some reason or re…

> My take is that rust is near the breaking point where it becomes so commonly used that more teams and organizations will just start to use it.

You would have said the same thing about PHP or Ruby (or even that they're past their breaking point), and yet companies that are stuck with either one today aren't too happy about it. The problem is that these days, there are a lot of people who can switch languages without much risk. They switch to X, and tomorrow they switch to Y. While it's very good to have low-risk early adopters, having so many of them can really give a false impression about long-term risk. At some point, you need to see many "long-term committers" making a switch.

Some people ask, but how can you have many long-term committers if these risk-averse people also look around for others like them? The answer is that usually this isn't the main factor. If some technology indeed makes a big bottom-line impact, there's a competitive risk in not adopting it (because your competitors will and then beat you in the market), which is why big-benefit technologies spread quickly.

Re: Being fair about memory safety and performance

#63

Post by _vvhw »

pcwalton wrote: post #51
_vvhw wrote: post #20

Earlier quoted context omitted.

> This article has a core point which is good: “in Rust the default is safe, and you have to opt-in to unsafety, but in C++ the default is unsafe, and you have to opt-in to safety”. On the subject of safe defaults, just to correct that Rust does not in fact have as much default memory safety with regards to buffer bleeds (e.g. variants of OpenSSL's Heartbleed) as it could [1], because it has unchecked arithmetic (int…

Do you have examples of this actually causing security problems in Rust code at anywhere near the levels of memory safety problems in C/C++? It strikes me as extremely dubious to tout Zig, which doesn't have memory safety at all, as somehow superior to Rust because Zig has this mitigation enabled by default (with a large performance cost) for an error class that Rust forestalls the vast majority of the negative conse…

> Do you have examples of this actually causing security problems in Rust

Do we need them though? I don't believe we need to go through a Heartbleed moment for Rust before we realize that checked arithmetic is just a good idea. We should be able to learn from the history of past exploits, even in other languages, not only in Rust.

I've also worked in the security industry. I've written static analysis software to detect zero-days. I've done professional bug bounty engagements. I know how to hack systems and I know from experience that checked arithmetic is important and something we should care more about as programmers.

Many programmers don't know about buffer bleeds and how dangerous are. There's a whole class of memory exploits that are much easier to pull off than a UAF. Security is about defense-in-depth. Why ship with an unsafe default?

> It strikes me as extremely dubious to tout Zig, which doesn't have memory safety at all

Comparing defaults with respect to checked arithmetic is something that programmers should be open to thinking and talking about. There's also no need to dismiss Zig's spatial memory safety as "no memory safety at all". Spatial memory safety is a pretty big win already. It rules out another class of exploits.

With respect to enabling checked arithmetic by default for safety, with an opt-out for performance at block scope level, Zig arguably has a safer default, something that I wish Rust would also adopt.

I don't agree that that's too "extremely dubious" to ask for?

> (with a large performance cost)

Not so. I'm working on a project that processes a million transactions a second.

We don't see any impact from this because the data plane is clearly delineated from the control plane. Checked arithmetic is enabled everywhere in fact because bounds checks are amortized across larger buffers. Compared to the cost of the cache misses to process the data, the costs of the arithmetic of the bounds check and the branch across the larger buffer are an order of magnitude less.

You can also opt-out at block scope level for hot loops. But we haven't needed to.

Again, projects should rather enable checked arithmetic by default for safety and then profile. Turning it off by default at the language layer with an opt-in for safety, in safe builds, just doesn't seem like the right default to me.

We can agree to disagree.

Re: Being fair about memory safety and performance

#64

Post by pron »

pron wrote: post #61
pcwalton wrote: post #53

Earlier quoted context omitted.

Zig is no way closer to Rust than C or C++ in this regard. Zig is in fact not appreciably safer than C or C++. All of the same classes of memory safety problems that are present in C or C++ are present in Zig. I am extremely doubtful of the claim that Zig eliminates 50% or more of memory safety problems.

As (safe) Zig eliminates overflows just as (safe) Rust does, and, like Rust, has not unsafe casts, I don't see how " all of the same classes of memory safety problems that are present in C or C++ are present in Zig." Also, Zig guarantees — just as Rust does — that all pointers and means of creating them are known, and precisely so, so analysis tools could work on Zig better than they do on C or C++. In C/C++, at best…

P.S.

I guess we could summarise Rust's and Zig's core design hypotheses as follows: Even though both place the same emphasis on correctness, Rust doesn't compromise on memory safety (which, given what empirical data we do have, is an important component of correctness but certainly not equivalent to it), i.e. it adds all the sound language features needed to provide it, even at the cost of language complexity, while Zig doesn't compromise on language simplicity, i.e. it adds all the sound language features needed to provide memory safety up to the point they impact language complexity. I don't discount the possibility that there might be a language that could be safer than Zig yet less complex than Rust, or perhaps even as soundly-safe as Rust and as simple as Zig, but so far I haven't seen such a language.

Barring any empirical data, we cannot say which, if any, of those two approaches leads to better correctness (where by "better" I mean reaching the desired level of correctness needed for most low-level applications more cheaply), so we both lean on "ideology," where I prefer simplicity whereas you prefer sound guarantees — both of us in the name of correctness. I think we agree that both C and Idris are the wrong paths to correctness, but while we might reasonably disagree on the price we should pay for soundness, placing Zig's memory-safety in the same category as C's is just as exaggerated and misleading as placing Rust's soundness in the same category as Idris's.

By the way, I wouldn't at all be surprised if empirical research ends up finding no significant differences in correctness between the two, and, in fact, would guess it to be the most likely outcome given our inability to find significant bottom-line differences between "reasonable" same-generation languages so far.

Re: Being fair about memory safety and performance

#65

Post by AndyKelley »

pcwalton wrote: post #57
dralley wrote: post #56

Earlier quoted context omitted.

The language by itself, perhaps not, I don't know. But, the fact that it has a test framework built in, and the test allocator fails when it detects memory problems (like a built-in valgrind) definitely puts it far above C in terms of making it easy to write correct code.

When I say that a language isn't meaningfully better than C in terms of memory safety, I mean C as programmed in 2022, which very much includes ASan.

You are misinformed. ASAN is a debugging tool. It is not a mechanism to provide safety to C applications.

Re: Being fair about memory safety and performance

#66

Post by dralley »

AndyKelley wrote: post #65
pcwalton wrote: post #57

Earlier quoted context omitted.

When I say that a language isn't meaningfully better than C in terms of memory safety, I mean C as programmed in 2022, which very much includes ASan.

You are misinformed. ASAN is a debugging tool. It is not a mechanism to provide safety to C applications.

I think their reply is fair as a response to the capabilities of "zig test"

Re: Being fair about memory safety and performance

#67

Post by pyjarrett »

pron wrote: post #61
pcwalton wrote: post #53

Earlier quoted context omitted.

Zig is no way closer to Rust than C or C++ in this regard. Zig is in fact not appreciably safer than C or C++. All of the same classes of memory safety problems that are present in C or C++ are present in Zig. I am extremely doubtful of the claim that Zig eliminates 50% or more of memory safety problems.

As (safe) Zig eliminates overflows just as (safe) Rust does, and, like Rust, has not unsafe casts, I don't see how " all of the same classes of memory safety problems that are present in C or C++ are present in Zig." Also, Zig guarantees — just as Rust does — that all pointers and means of creating them are known, and precisely so, so analysis tools could work on Zig better than they do on C or C++. In C/C++, at best…

> Rust easily makes the top four most complex "production" languages in history (together with C++, Ada, and Scala)

Ada was "complicated" when it was released because it was being compared to C. Contrasting it against contemporary C++ or Rust, Ada 2012 is much simpler.

Re: Being fair about memory safety and performance

#68

Post by pcwalton »

AndyKelley wrote: post #65
pcwalton wrote: post #57

Earlier quoted context omitted.

When I say that a language isn't meaningfully better than C in terms of memory safety, I mean C as programmed in 2022, which very much includes ASan.

You are misinformed. ASAN is a debugging tool. It is not a mechanism to provide safety to C applications.

We're talking about tests here.

I work with ASan for a living. I know what it is, thank you :)

Re: Being fair about memory safety and performance

#69

Post by pcwalton »

_vvhw wrote: post #63
pcwalton wrote: post #51

Earlier quoted context omitted.

Do you have examples of this actually causing security problems in Rust code at anywhere near the levels of memory safety problems in C/C++? It strikes me as extremely dubious to tout Zig, which doesn't have memory safety at all, as somehow superior to Rust because Zig has this mitigation enabled by default (with a large performance cost) for an error class that Rust forestalls the vast majority of the negative conse…

> Do you have examples of this actually causing security problems in Rust Do we need them though? I don't believe we need to go through a Heartbleed moment for Rust before we realize that checked arithmetic is just a good idea. We should be able to learn from the history of past exploits, even in other languages, not only in Rust. I've also worked in the security industry. I've written static analysis software to det…

The data clearly show a significant overhead for integer bounds checks. https://danluu.com/integer-overflow/

At scale this adds up to millions of dollars. It would be a deal breaker for switching to Rust for many organizations. That's why bounds checks in release mode are not the default.

Re: Being fair about memory safety and performance

#70

Post by _vvhw »

pcwalton wrote: post #69
_vvhw wrote: post #63

Earlier quoted context omitted.

> Do you have examples of this actually causing security problems in Rust Do we need them though? I don't believe we need to go through a Heartbleed moment for Rust before we realize that checked arithmetic is just a good idea. We should be able to learn from the history of past exploits, even in other languages, not only in Rust. I've also worked in the security industry. I've written static analysis software to det…

The data clearly show a significant overhead for integer bounds checks. https://danluu.com/integer-overflow/ At scale this adds up to millions of dollars. It would be a deal breaker for switching to Rust for many organizations. That's why bounds checks in release mode are not the default.

> At scale this adds up to millions of dollars.

Buffer bleeds like Heartbleed cost the industry hundreds of millions of dollars.

Also, to be fair, Dan's post is about checked arithmetic in hot loops, i.e. the data plane, which as I've said, "large organizations" would know to amortize by using large buffers, and by clearly delineating between data plane and control plane.

For example, why not simply disable checked arithmetic at block scope level for a hot loop? Disabling at program level by default, and then having to re-enable it everywhere that's not a hot loop, just seems like conflating data plane and control plane, and like a massive overly big hammer.

It's also dangerous, because unsafe defaults might be run by new programmers who don't understand the risks of unchecked arithmetic, and who think that Rust gives them 100% memory safety.

Also, do you think that a 5% penalty on control planes is cost-prohibitive? I don't.

Control planes are usually where "large organizations" have tons of assertions anyway, for example, AWS really like to run their control planes at constant max load, regardless of actual load, to avoid cascading failure. That costs them millions of dollars, but relative to the hundreds of millions of dollars that their data planes cost, it's worth it, because it saves outages and failures that could easily dwarf the 5% performance gains at the expense of safety.

Safety becomes much more critical at large scale in fact, more so than performance. Better to be correct first, and then fast. Than fast, but not correct.

Post reply on HN