Unsafe Rust: An Intro and Open Questions • Hacker News Board

Re: Unsafe Rust: An Intro and Open Questions

#21

Post by rcxdude » Thu, May 28, 2015, 9:38 PM UTC

eslaught wrote: post #13

dschatz wrote: post #11

Earlier quoted context omitted.

Just to add to this point, the difference is that Rust can only guarantee this for the standard library. I can similarly write a library with safe interfaces that can be (ab)used to cause UB and there's little that the Rust team can do. This is different from other "safe" languages. This is why it's so important to establish what the responsibility and expectation is of library developers to uphold the safety guarant…

This is not all that different from Java (or Python, etc.), where it is quite easy to hide a call to a native function behind a seemingly-safe interface. The real difference is that native methods in Java must be written in a different language (C), while Rust supports both modes in the same language. (Edit: Or, if you prefer, two different but very closely related languages.) I would argue, at any rate, that this so…

Indeed. I have frequently segfaulted python by using certain modules (not even particularly obscure or low-quality ones either).

Re: Unsafe Rust: An Intro and Open Questions

#22

Post by Jweb_Guru » Thu, May 28, 2015, 10:29 PM UTC

Animats wrote: post #16

"Unsafe" is an escape hatch used for a number of reasons. There are good ones and bad ones. Bad ones include: - "I'm so l33t I don't need the compiler to check me." (Don't hire those guys.) - "Safe code is too slow". (File bugs on the compiler's optimizer.) - "Porting this to safe code would require a redesign". (See the Rust port of DOOM.) Most of the real needs for "Unsafe" in Rust come from - The need to interface…

> It's a good candidate for formal proof of correctness - not too big, and critical to system operation.

Unfortunately, having actually looked for production strength proofs of correctness for production allocators (as opposed to toys), I wasn't able to find any. Given how high-value of a target they are for formal verification, we may be underestimating the complexity of doing so.

Re: Unsafe Rust: An Intro and Open Questions

#23

Post by krick » Thu, May 28, 2015, 10:43 PM UTC

sanderjd wrote: post #17

gregwtmtno wrote: post #14

Earlier quoted context omitted.

Right, but if you write a library using only Rust "safe" code, the guarantee is back on the Rust team. To me, it would be better if libraries using unsafe code were marked.

Here's[0] an interesting idea for forcing crates using unsafe code to be handled specially, while allowing some "blessed" crates through without the special handling. [0]: https://github.com/rust-lang/cargo/issues/934#issuecomment-6...

That seems like absolutely great idea. I want it already. I wonder why it wasn't processed further, while being 6 month old already.

It is different notion of "blessed" than the original proposal and, IMHO, a much better one. Steve's proposal is quite dubious, I guess. The problem is low standard for the word "blessed" here: in his proposal this badge has no technical meaning, but big social impact. Crate doesn't have to be superior in a technical sense to get this badge, it just has to be "famous", and stuff gets "famous" for various reasons. That's really bad and will get worse when Rust/Cargo will become more popular.

But that "safe/unsafe" isn't a matter of opinion anymore. If a library is known to be unsafe through the "safe" interface: it's a bug, and a crate shouldn't have this badge from a moment the bug has been discovered and until the bug is fixed again (or even longer, if there have been 5 such bugs over the last 2 weeks, even though now they seem to be fixed). It somewhat serves the original purpose (I assume), because it still means that a crate that isn't used heavily enough ("was downloaded N times over the last month") cannot be "blessed" — we don't have all necessary information to mark it as "blessed" yet, so it will help to set "junk crates" aside.

It's worth nothing if a library is made by Github, but is known to be buggy and to leak memory. But it is worth something if a "github API" crate made by John Doe is used by hundreds of people and didn't have a single memory leak for quite a while.

Re: Unsafe Rust: An Intro and Open Questions

#24

Post by steveklabnik » Thu, May 28, 2015, 11:22 PM UTC

krick wrote: post #23

sanderjd wrote: post #17

Earlier quoted context omitted.

Here's[0] an interesting idea for forcing crates using unsafe code to be handled specially, while allowing some "blessed" crates through without the special handling. [0]: https://github.com/rust-lang/cargo/issues/934#issuecomment-6...

That seems like absolutely great idea. I want it already. I wonder why it wasn't processed further, while being 6 month old already. It is different notion of "blessed" than the original proposal and, IMHO, a much better one. Steve's proposal is quite dubious, I guess. The problem is low standard for the word "blessed" here: in his proposal this badge has no technical meaning, but big social impact. Crate doesn't hav…

> Steve's proposal is quite dubious, I guess.

I pretty much opened this issue so we could talk about it, not because I think it's a good idea.

Re: Unsafe Rust: An Intro and Open Questions

#25

Post by krick » Thu, May 28, 2015, 11:54 PM UTC

steveklabnik wrote: post #24

krick wrote: post #23

Earlier quoted context omitted.

That seems like absolutely great idea. I want it already. I wonder why it wasn't processed further, while being 6 month old already. It is different notion of "blessed" than the original proposal and, IMHO, a much better one. Steve's proposal is quite dubious, I guess. The problem is low standard for the word "blessed" here: in his proposal this badge has no technical meaning, but big social impact. Crate doesn't hav…

> Steve's proposal is quite dubious, I guess. I pretty much opened this issue so we could talk about it, not because I think it's a good idea.

Oh, I didn't mean any offense. I'm just commenting on the idea: the original wording of the proposal seems dangerous, but both reem's and yazaddaruvala's ideas definitely have potential.

So you did a good job by starting that discussion. I hope it will have results.

Re: Unsafe Rust: An Intro and Open Questions

#26

Post by steveklabnik » Fri, May 29, 2015, 12:16 AM UTC

krick wrote: post #25

steveklabnik wrote: post #24

Earlier quoted context omitted.

> Steve's proposal is quite dubious, I guess. I pretty much opened this issue so we could talk about it, not because I think it's a good idea.

Oh, I didn't mean any offense. I'm just commenting on the idea: the original wording of the proposal seems dangerous , but both reem's and yazaddaruvala's ideas definitely have potential. So you did a good job by starting that discussion. I hope it will have results.

It's all good, none taken! :)

Re: Unsafe Rust: An Intro and Open Questions

#27

Post by dbaupp » Fri, May 29, 2015, 12:53 AM UTC

Animats wrote: post #16

"Unsafe" is an escape hatch used for a number of reasons. There are good ones and bad ones. Bad ones include: - "I'm so l33t I don't need the compiler to check me." (Don't hire those guys.) - "Safe code is too slow". (File bugs on the compiler's optimizer.) - "Porting this to safe code would require a redesign". (See the Rust port of DOOM.) Most of the real needs for "Unsafe" in Rust come from - The need to interface…

> The first one is mostly a problem with expressive power in the foreign function interface. Can you express what "int read(int fd, char buf[], size_t len)" means in the foreign function definition syntax? Rust's foreign function syntax isn't expressive enough to do that.[1] You can't tell Rust that "len" is the length of "buf". Being able to do that would help reduce the need for unsafe code. Most of the POSIX/Linux API can be described with relatively simple syntax that allows you to associate size info with C arrays. (I once proposed this as an extension to C. It's technically possible but politically too difficult.)[2]

There's still a need for `unsafe`, since it's possible for the relationship to be described incorrectly. It's fundamentally not something the compiler can check, and hence requires `unsafe` conceptually (if not in practice).

One can regard wrapping FFI functions in safe interfaces as specifying the relationships between parameters.

> If you cast 4 bytes to a 32-bit unsigned integer, the result is always a valid 32-bit unsigned integer. Conversions like that should be explicit, but are not memory-unsafe.

Only a very small subset of types have the property that any bit-pattern is safe, essentially only primitives. So this seems like a rather limited way to reduce unsafety (instead of just writing a short library function once).

Re: Unsafe Rust: An Intro and Open Questions

#28

Post by TheLoneWolfling » Fri, May 29, 2015, 1:20 AM UTC

> with the caveat that main can't be unsafe

Why not? Is there a specific justification for this beyond just "you shouldn't do that"?

Re: Unsafe Rust: An Intro and Open Questions

#29

Post by Animats » Fri, May 29, 2015, 2:55 AM UTC

dbaupp wrote: post #27

Animats wrote: post #16

"Unsafe" is an escape hatch used for a number of reasons. There are good ones and bad ones. Bad ones include: - "I'm so l33t I don't need the compiler to check me." (Don't hire those guys.) - "Safe code is too slow". (File bugs on the compiler's optimizer.) - "Porting this to safe code would require a redesign". (See the Rust port of DOOM.) Most of the real needs for "Unsafe" in Rust come from - The need to interface…

> The first one is mostly a problem with expressive power in the foreign function interface. Can you express what "int read(int fd, char buf[], size_t len)" means in the foreign function definition syntax? Rust's foreign function syntax isn't expressive enough to do that.[1] You can't tell Rust that "len" is the length of "buf". Being able to do that would help reduce the need for unsafe code. Most of the POSIX/Linux…

"Only a very small subset of types have the property that any bit-pattern is safe, essentially only primitives."

Structs which contain only primitives have that property. Consider a TCP header.

Re: Unsafe Rust: An Intro and Open Questions

#30

Post by Animats » Fri, May 29, 2015, 2:59 AM UTC

Jweb_Guru wrote: post #22

Animats wrote: post #16

"Unsafe" is an escape hatch used for a number of reasons. There are good ones and bad ones. Bad ones include: - "I'm so l33t I don't need the compiler to check me." (Don't hire those guys.) - "Safe code is too slow". (File bugs on the compiler's optimizer.) - "Porting this to safe code would require a redesign". (See the Rust port of DOOM.) Most of the real needs for "Unsafe" in Rust come from - The need to interface…

> It's a good candidate for formal proof of correctness - not too big, and critical to system operation. Unfortunately, having actually looked for production strength proofs of correctness for production allocators (as opposed to toys), I wasn't able to find any. Given how high-value of a target they are for formal verification, we may be underestimating the complexity of doing so.

Successes have been claimed.[1]

It's discouraging to me how little progress there's been in proof of correctness in the last 35 years. I used to work on that stuff. C set the field back by decades.

[1] https://books.google.com/books?id=yvJqCQAAQBAJ&pg=PA364&lpg=...