Rust is mostly safety • Hacker News Board

Re: Rust is mostly safety

#351

Post by dtolnay » Fri, Dec 30, 2016, 1:56 AM UTC

Animats wrote: post #302

Manishearth wrote: post #283

Earlier quoted context omitted.

I just gave you backup; I grepped my whole .cargo cache dir (both the one used by servo and my global one). You have also made your claim without backup -- you have repeatedly claimed that this is an endemic problem in Rust, with only individual crates (most of them obscure ones) to back it up, and I only usually make my claim in response to claims like yours -- the burden of proof is on you. Anyway, I do provide som…

I looked at a few. "itoa" is clearly premature optimization. That uses an old hack appropriate to machines where integer divide was really expensive, like an Arduino-class CPU. It's unlikely to help much on anything with a modern divide unit. "httpparse", "idna", "serde-json", and "inflate" should be made 100% safe - they all take external input, are used in web-facing programs, and are classic attack vectors. Not mu…

I maintain two of the crates you called out so here is a bit more detail on the use cases:

"itoa" is code that is copied directly from the Rust core library. Every character of unsafe code is identical to what literally everybody who uses Rust is already running (including people using no_std). Anybody who has printed an integer in Rust has run the same unsafe code. It is some of the most widely used code in Rust. If I had rewritten any of it, even using entirely safe code, it would be astronomically more likely to be wrong than copying the existing code from Rust. The readme contains a link to the exact commit and block of code from which it is copied.

As for premature optimization, nope it was driven by a very standard (across many languages) set of benchmarks: https://github.com/serde-rs/json-benchmark

"serde_json" uses an unsafe assumption that a slice of bytes is valid UTF-8 in two places. This is either for performance or for maintainability, depending on how you look at it. Performance is the more obvious reason but in fact we could get all the same speed just by duplicating most of the code in the crate. We support deserializing JSON from bytes or from a UTF-8 string, and we support serializing JSON to bytes or to a UTF-8 string. Currently these both go through the same code path (dealing with bytes) with an unchecked conversion in two important spots to handle the UTF-8 string case. One of those cases takes advantage of the assumption that if the user gave us a &str, they are guaranteeing it is valid UTF-8. The other case is taking advantage of the knowledge that JSON output generated by us is valid UTF-8 (which is checked along the way as it is produced).

Here again, both of those uses are driven by the benchmarks in the repo above and account for a substantial performance improvement over a checked conversion.

Re: Rust is mostly safety

#352

Post by rcthompson » Fri, Dec 30, 2016, 2:00 AM UTC

zerofan wrote: post #333

rcthompson wrote: post #304

Earlier quoted context omitted.

No, I haven't done any skydiving. After writing my comment, I suspected my analogy might not hold up if I knew more about skydiving. I guess just imagine an abstract form of skydiving where you just need to have fun in the sky and then open your chute at the right altitude, and you'd like to wait as long as you can before opening it. I still think the point is valid, though, even if the analogy isn't.

Fair enough. I'm sorry about calling you out on the metaphor. Instead I'll call you out on the point itself :-) > Having safety features that you know you can rely on allows you to take risks that you normally wouldn't in order to accomplish some really awesome things. Unless you rush to publish a public facing version of your code, I can't see why you'd be afraid to take risks in any language. What's so scary about…

Perhaps a better analogy would be the way that much of modern medicine is enabled by access to antibiotics. Without antibiotics, the risk of post-operation death by infection would be so high as to rule out many of the procedures that we now consider safe and routine.

Re: Rust is mostly safety

#353

Post by Animats » Fri, Dec 30, 2016, 2:03 AM UTC

dtolnay wrote: post #351

Animats wrote: post #302

Earlier quoted context omitted.

I looked at a few. "itoa" is clearly premature optimization. That uses an old hack appropriate to machines where integer divide was really expensive, like an Arduino-class CPU. It's unlikely to help much on anything with a modern divide unit. "httpparse", "idna", "serde-json", and "inflate" should be made 100% safe - they all take external input, are used in web-facing programs, and are classic attack vectors. Not mu…

I maintain two of the crates you called out so here is a bit more detail on the use cases: "itoa" is code that is copied directly from the Rust core library. Every character of unsafe code is identical to what literally everybody who uses Rust is already running (including people using no_std). Anybody who has printed an integer in Rust has run the same unsafe code. It is some of the most widely used code in Rust. If…

"serde_json" uses an unsafe assumption that a slice of bytes is valid UTF-8 in two places. This is either for performance or for maintainability, depending on how you look at it. Performance is the more obvious reason but in fact we could get all the same speed just by duplicating most of the code in the crate.

Could that be done safely with a generic, instantiated for both types?

Re: Rust is mostly safety

#354

Post by karllager » Fri, Dec 30, 2016, 2:10 AM UTC

gmjosack wrote: post #198

nemo1618 wrote: post #186

Earlier quoted context omitted.

Go has one very significant advantage over Rust: Simplicity. Reading someone else's Go code is such a breath of fresh air compared to reading someone else's C++ code (or god forbid, Haskell). Having a standard formatting further enhances this. I feel like this aspect of Go is not given enough weight. It's a huge benefit to large software projects and large software companies. I also suspect that this may be why the G…

In my experience working with new and experienced Go developers that simplicity is pretty superficial. Take a look at SliceTricks[0]. These are all pretty simple operations in other languages that the Go authors think users should be forced to write manually so they understand the costs. Reading these in code reviews is definitely not a breath of fresh air. I also see Go users develop bad habits that will burn them i…

Go is just the current incarnation of the so-called New Jersey approach.

> I believe that worse-is-better, even in its strawman form, has better survival characteristics than the-right-thing.

Re: Rust is mostly safety

#355

Post by zerofan » Fri, Dec 30, 2016, 2:15 AM UTC

rcthompson wrote: post #350

zerofan wrote: post #333

Earlier quoted context omitted.

Fair enough. I'm sorry about calling you out on the metaphor. Instead I'll call you out on the point itself :-) > Having safety features that you know you can rely on allows you to take risks that you normally wouldn't in order to accomplish some really awesome things. Unless you rush to publish a public facing version of your code, I can't see why you'd be afraid to take risks in any language. What's so scary about…

I think it's more like: since you know the compiler won't let you write a buffer overflow, use-after-free, data race, etc., you no longer have to waste time worrying about whether your code might contain such problems, which frees up more mental bandwidth for other concerns. But unlike other languages, you still have confidence that your code will compile to equally performant machine code (e.g. No GC overhead). The…

> I think it's more like: since you know the compiler won't let you write a buffer overflow, use-after-free, data race, etc., you no longer have to waste time worrying about whether your code might contain such problems, which frees up more mental bandwidth for other concerns.

I've spent a lot of time figuring out how to do completely mundane things in Rust. At this point, buffer overflows and use-after-frees are not my biggest concerns in C++.

> The scary thing isn't causing a segfault on your local machine. The scary thing is writing code that could segfault but doesn't do in testing until after you've deployed it publicly. If your compiler rejects code that can segfault, this is no longer a concern.

If your testing didn't catch the problem (which I can fully understand), a panic at runtime is not much different than a segfault.

> (Or replace segfault with a buffer overflow that leaks your private keys or something equally bad.)

I firmly believe the OpenSLL team would've used unsafe blocks in Rust to disable the performance overhead of bounds checking. That whole exploit was caused by sloppy optimizations, and Rust is not immune from that.

Re: Rust is mostly safety

#356

Post by mmstick » Fri, Dec 30, 2016, 2:23 AM UTC

jwn wrote: post #305

kibwen wrote: post #263

Earlier quoted context omitted.

I think this comment and the OP are both correct. The point of Rust is safety, in the sense that memory safety should be the default for all programming languages, and has been the default for all non-systems languages since the 90s. The only holdouts have been because people used to claim that memory safety wasn't possible without making a language unusably slow, which Rust has disproven. In all my years of teaching…

I really want to love Rust, and while I understand the Rust borrow checker in theory, actually using it in practice has been a major headache. I tried Rust on a simple terminal-based project, and after a week of feeling that I was getting nowhere I switched to Go and had a proof of concept in several hours. With that said, can you recommend a good source to really understand best practices and patterns for ownership…

Mind sharing the source code? I can whip out a Rust version in the same amount of time. I have been using Rust for two years. I never worry about the borrow checker as it never gives me problems.

Re: Rust is mostly safety

#357

Post by zerofan » Fri, Dec 30, 2016, 2:28 AM UTC

rcthompson wrote: post #352

zerofan wrote: post #333

Earlier quoted context omitted.

Fair enough. I'm sorry about calling you out on the metaphor. Instead I'll call you out on the point itself :-) > Having safety features that you know you can rely on allows you to take risks that you normally wouldn't in order to accomplish some really awesome things. Unless you rush to publish a public facing version of your code, I can't see why you'd be afraid to take risks in any language. What's so scary about…

Perhaps a better analogy would be the way that much of modern medicine is enabled by access to antibiotics. Without antibiotics, the risk of post-operation death by infection would be so high as to rule out many of the procedures that we now consider safe and routine.

I would prefer a surgeon who washed his hands over one who didn't but gave me antibiotics. I've had stitches a few times, but only one real surgery. I never got antibiotics for any of those. Maybe we could skip the analogies? I don't think they help the discussion.

Re: Rust is mostly safety

#358

Post by mmstick » Fri, Dec 30, 2016, 2:29 AM UTC

dom0 wrote: post #134

agentgt wrote: post #94

I'm a lowly ancient Java programmer and I think Rust is far far more than safety. In my opinion Rust is about doing things right. It may have been about safety at first but I think it is more than that given the work of the community. Yes I know there is the right tool for the right job and is impossible to fill all use cases but IMO Rust is striving for iPhone like usage. I have never seen a more disciplined and bal…

> In my opinion Rust is about doing things right. On the other hand there is a quite dark cloud on the horizon with the stable vs nightly split. You can't run infrastructure on nightly builds; or add nightly builds to distributions.

> add nightly builds to distributions

rustup run nightly cargo build --release

You were saying? In any case, there's no such thing as a stable vs nightly split. There's pretty much zero libraries that require nightly, and the few that have a nightly option for optional features will no longer require nightly after the macros update lands.

Re: Rust is mostly safety

#359

Post by hedora » Fri, Dec 30, 2016, 2:32 AM UTC

cmyr wrote: post #233

jfaucett wrote: post #187

Earlier quoted context omitted.

IMO the biggest plus for rust is actually cargo. Building, versioning, and sharing modular code is essentially copy/paste in C/C++, and compared to that cargo is lightyears ahead. I actually wish rust would accept its systems niche even more and move the stdlib to crates and make nostd the default mode. Personally, I see no reason to market rust for webapps or gui stuff, it cant/wont compete with Rails/QT for years t…

I couldn't agree more with this. Cargo (and the general desire to think and work hard on ease-of-use) is a huge part of what makes Rust a pleasure. It alone would be enough for me to steer someone with experience in neither to Rust over C++ for lots and lots of use cases.

I haven't looked at rust, and this sentiment is exactly what has kept me away. I watched perl, java, python, ruby, node and c++ (boost) fall into the trap of "we know better than the end-user/developers/sysadmins/os vendor, so let's reinvent dpkg poorly".

Why should cargo be any different? It is solving a problem I don't have (debian, ubuntu, openbsd, and freaking illumnos all have acceptable package management), and creating a massive new problem (there is a whole thread below this one talking about rust dll hell between nightly and stable, and the thread links to other HN articles!). From my perspective all this work is wasted just because some developers somewhere use an OS that doesn't support apt or ports.

Sorry this is so ranty, but I really want to know if anyone has had luck using rust with their native package manager.

Re: Rust is mostly safety

#360

Post by dtolnay » Fri, Dec 30, 2016, 2:42 AM UTC

Animats wrote: post #353

dtolnay wrote: post #351

Earlier quoted context omitted.

I maintain two of the crates you called out so here is a bit more detail on the use cases: "itoa" is code that is copied directly from the Rust core library. Every character of unsafe code is identical to what literally everybody who uses Rust is already running (including people using no_std). Anybody who has printed an integer in Rust has run the same unsafe code. It is some of the most widely used code in Rust. If…

"serde_json" uses an unsafe assumption that a slice of bytes is valid UTF-8 in two places. This is either for performance or for maintainability, depending on how you look at it. Performance is the more obvious reason but in fact we could get all the same speed just by duplicating most of the code in the crate. Could that be done safely with a generic, instantiated for both types?

Yes, that is what we already do. The two unsafe UTF-8 casts are the two critical spots at opposite edges of the generic abstraction where the instantiation corresponding to UTF-8 string needs to take advantage of the knowledge that something is guaranteed to be valid UTF-8.

What we have is as close as possible to what you suggested.

As I mentioned, we could get rid of the unsafe code in other ways without sacrificing performance. Ultimately it is up to me as a maintainer of serde_json to judge the likelihood and severity of certain types of bugs and make tradeoffs appropriately. There are security-critical bugs we could implement using only safe code, for example if you give us JSON that says {"user": "Animats"} and we deserialize it as {"user": "admin"}. My judgment is that using 100% safe code would increase the likelihood of other types of bugs (not related to UTF-8ness) and the current tradeoff is what makes the most sense for the library.

From another point of view, performance and safety are synonyms in this case, not opposites. If we use 0.1% unsafe code and perform faster than the fastest 100% unsafe C/C++ library (which is what the benchmarks show for many use cases) then people will be inclined to use our 0.1% unsafe library. If we give up unsafe but sacrifice performance, people will be inclined to use the 100% unsafe C/C++ alternatives.