Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bugs You'll Probably Only Have in Rust

gankro.github.io

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Bugs You'll Probably Only Have in Rust

#31

Post by k__ »

steveklabnik wrote: post #19
halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

I'll give you the shortest example: in order to build an operating system in Rust for x86, you need to do this: let p = 0xb8000 as *mut u8; VGA drivers use the memory mapped at 0xb8000 to drive the device. This creates a pointer, p, at that address. In order to demonstrate this is safe (okay so unsafe isn't in this example, creating p is safe, but writing to/reading from it is not), a language would have to know: 1.…

Can't the prohibitions be modularized?

Like, when you compile for x86 there are a bunch of rules that aren't generally safe, but on that platform they are.

Re: Bugs You'll Probably Only Have in Rust

#32

Post by jeeyoungk »

halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

It also depends on how "deep" you want to get.

A lot of built-in constructs uses unsafe under the hood. Vector (Rust's dynamic arrays) does memory allocation / resizing under the hood for example, and there's no safe way to do it, unless some safer array allocation primitive is exposed.

Also things like mem::swap(x, y) cannot be implemented at all with safe rust. in order to perform swap, you need a temporary variable. That temporary variable would be uninitialized, which Rust does not allow.

Note that in c++ it invokes copy constructor - http://www.cplusplus.com/reference/algorithm/swap/ - but Rust's mem::swap works for types that does not implement the Copy trait (Rust's equivalent to copy constructor).

Same can be said about slice splitting functions, which is primarily used to work around Rust's borrow checker.

Re: Bugs You'll Probably Only Have in Rust

#33

Post by steveklabnik »

jacquesm wrote: post #30
steveklabnik wrote: post #28

Earlier quoted context omitted.

The language would still have to understand all of that in order to write that kernel side allocator in safe code.

I don't see how that follows. The language can't possibly understand the intricacies of what the MMU is capable of (besides, every MMU is different), and as far as the language is concerned what is returned is simply a valid offset and a length to go with it to indicate where the allocated segment ends.

I think you're strongly agreeing with me. It's not feasible to have in the language.

Re: Bugs You'll Probably Only Have in Rust

#34

Post by steveklabnik »

k__ wrote: post #31
steveklabnik wrote: post #19

Earlier quoted context omitted.

I'll give you the shortest example: in order to build an operating system in Rust for x86, you need to do this: let p = 0xb8000 as *mut u8; VGA drivers use the memory mapped at 0xb8000 to drive the device. This creates a pointer, p, at that address. In order to demonstrate this is safe (okay so unsafe isn't in this example, creating p is safe, but writing to/reading from it is not), a language would have to know: 1.…

Can't the prohibitions be modularized? Like, when you compile for x86 there are a bunch of rules that aren't generally safe, but on that platform they are.

Modularization wouldn't help the fact that you'd still need a module per platform and that's not feasible, see the other replies to my comment. There's just far, far, far too many details.

Re: Bugs You'll Probably Only Have in Rust

#35

Post by eddyb »

steveklabnik wrote: post #16
cpeterso wrote: post #15

Earlier quoted context omitted.

> 3. Accessing or modifying a mutable static variable (and this might conceivably even be removed entirely someday) Mutable static variables removed or the unsafety of accessing them? Didn't Rust, at one early point, not allow mutable global variables?

One of the compiler team members advocated for removing static mut entirely, but it didn't quite happen before 1.0. It's totally feasible to do so if const fn was stabilized, but it's not, so...

:(

Re: Bugs You'll Probably Only Have in Rust

#36

Post by eru »

wyldfire wrote: post #3

> Making unsafe a big scary "all bets are off" button is only compelling if most of our users don't need to use that button. Rust is trying to be a language for writing concurrent applications, so sharing your type between threads requiring unsafe would be really bad. It would be neat if we could decompose unsafe like so "unsafe[this_feature,that_feature] {}". The unqualified "unsafe" could still refer to a global "f…

At Standard Chartered our in-house Haskell dialect did a tiny bit of that: it distinguishes between ReadIO and general IO. ReadIO is meant to be idempotent operations only.

Re: Bugs You'll Probably Only Have in Rust

#37

Post by Manishearth »

Animats wrote: post #23
halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization. There's a lot…

I assume "rust fanboy" is me.

Please stop it. This is the fourth time you're making this unsubstantiated claim.

Yes, I backed up my argument with "the crates I am using" (that is, all the crates in the entire dependency tree of any crate I've used in the last year), which is at least somewhat representative of the crates that folks use, as opposed to your wild claims that have zero backup (or are backed up with one or two mentions of crates, where subsequently the use of unsafe is justified by me or others).

Here's an example of one of these discussions: https://news.ycombinator.com/item?id=14276129 . Here's another: https://news.ycombinator.com/item?id=13280347 . In the second one I actually admit that I was surprised at the number of crates doing unchecked indexing, that's definitely not denial. I still maintained there that it was quite manageable and not some crazy pervasive problem as you make it out to be, but you've literally never provided evidence otherwise. That's not denial.

As I've mentioned before, I'm genuinely interested in finding these cases so that they can be fixed (or replaced with well-audited general purpose unsafe abstraction libraries -- e.g. a cheap ASCII-within-utf8 handling library that lets you do things like munch ascii bytes while parsing without extra overhead (or whatever the use cases are -- I'm not yet clear on the set of use cases out there which is why I'm interested to hear more). I'm not "denying everything", and other people have even pointed this out (https://news.ycombinator.com/item?id=14277634) in the discussions we've had.

-----

FWIW, IIRC UTF8 stuff has been a perf bottleneck in the url crate. It's not premature (file a bug, though). That said, we might be able to write that code differently to avoid it or have a more general way of doing it with clearer invariants (I'll think about it!).

The ordermap crate does implement a hash map without unsafe code (leveraging vec). The stdlib hashmap uses unsafe because its design (which has different tradeoffs wrt ordermap) needs to be able to have buckets be empty (where whether or not the bucket is empty is based on a nontrivial set of invariants from the robin hood based design) without using something like an Option which has extra memory and CPU overhead. Optimizers are not smart enough to figure out the robin hood invariants, so this is not premature either.

----

I no longer believe you're making these claims in good faith, so I'm not really interested in discussing this further with you. I just want to counter the wild claims here for the benefit of other folks reading this.

If other folks wish to constructively point out cases of unsafe being used this way, I'd love to hear about it, both to get a better idea of the ecosystem, and to help improve it. I do want the community to move towards less unsafe code.

Re: Bugs You'll Probably Only Have in Rust

#38

Post by Animats »

Manishearth wrote: post #37
Animats wrote: post #23

Earlier quoted context omitted.

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization. There's a lot…

I assume "rust fanboy" is me. Please stop it. This is the fourth time you're making this unsubstantiated claim. Yes, I backed up my argument with "the crates I am using" (that is, all the crates in the entire dependency tree of any crate I've used in the last year), which is at least somewhat representative of the crates that folks use, as opposed to your wild claims that have zero backup (or are backed up with one o…

Note lack of links to code above.

Please provide a list of the basic Rust crates which contain no unsafe code. Can you write useful programs using only those crates, not counting ones which need foreign function calls to talk to the OS?

Re: Bugs You'll Probably Only Have in Rust

#39

Post by burntsushi »

Manishearth wrote: post #37
Animats wrote: post #23

Earlier quoted context omitted.

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization. There's a lot…

I assume "rust fanboy" is me. Please stop it. This is the fourth time you're making this unsubstantiated claim. Yes, I backed up my argument with "the crates I am using" (that is, all the crates in the entire dependency tree of any crate I've used in the last year), which is at least somewhat representative of the crates that folks use, as opposed to your wild claims that have zero backup (or are backed up with one o…

> IIRC UTF8 stuff has been a perf bottleneck in the url crate

Right, and this isn't surprising at all. It's not at all obvious to me that it's premature. It's really easy for from_utf8 to show up in a profile, and I've had occasion to either work around it safely (perhaps by staying in `&[u8]` land) or by resorting to `unsafe`. I did the latter in the CSV crate for managing `StringRecord`s.[1] The record is stored contiguously in memory, which means that even if the entire record is valid UTF-8 as a whole, then each individual field may not be. Checking UTF-8 validity on each field or on access leads to a slow down. (There are benchmarks.) It's much nicer to code a fast path that checks if the entire record is all ASCII. (Because then you can assume each field is valid UTF-8.)

[1] - https://github.com/BurntSushi/rust-csv/blob/34e957e63bb92853...

Re: Bugs You'll Probably Only Have in Rust

#40

Post by openasocket »

Animats wrote: post #38
Manishearth wrote: post #37

Earlier quoted context omitted.

I assume "rust fanboy" is me. Please stop it. This is the fourth time you're making this unsubstantiated claim. Yes, I backed up my argument with "the crates I am using" (that is, all the crates in the entire dependency tree of any crate I've used in the last year), which is at least somewhat representative of the crates that folks use, as opposed to your wild claims that have zero backup (or are backed up with one o…

Note lack of links to code above. Please provide a list of the basic Rust crates which contain no unsafe code. Can you write useful programs using only those crates, not counting ones which need foreign function calls to talk to the OS?

> Note lack of links to code above.

He provided links to several other threads where he explicitly has made a list. In this one https://news.ycombinator.com/item?id=13280347 he says he looked at ~600 crates he actually uses, and found 70 used unsafe code. Those 70 are given here: https://gist.github.com/Manishearth/6a9367a7d8772e095629e821... with an explanation of how and why they use unsafe. In that comment, he even addresses that he's suprised the number is that high and that work can be done to lower it. You are seriously misrepresenting his contributions to this discussion.

Post reply on HN