Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bugs You'll Probably Only Have in Rust

gankro.github.io

Post reply on HN
Print view View original

21–30 of 91 posts

Re: Bugs You'll Probably Only Have in Rust

#21

Post by dbaupp »

halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

C and other FFI is fundamentally outside the control of the Rust compiler (or any other non-C compiler) and, furthermore, the foreign functions can have arbitrarily complicated preconditions (or plain bugs) that lead to memory safety violations. This means that, whether it is marked or not, these operations are semantically `unsafe`, as in, they risk memory unsafety because the compiler can't guarantee it. This does mean that all languages with FFI have safety holes.

Additionally, not having the facility for low-level/unchecked code just means that things like optimised data structures/memory management/hardware interaction get implemented either in the compiler or in other languages. The former is much harder to reason about and to modify: one is essentially writing code that generates compiler IR, which is more annoying and error prone that both just writing the code directly and just writing the IR directly (one way to think about this is the compiler is one big `unsafe` block). The latter is unfortunate because it results in impedence mismatches when doing the FFI calls both semantically and with performance, and it also means that code doesn't get to benefit from the usual Rust safe checks and high level features (like ADTs) that are all still available inside `unsafe` blocks.

Re: Bugs You'll Probably Only Have in Rust

#22

Post by Gankro »

halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

Do you mean:

* All unsafe operations don't exist.

* All unsafe operations exist, but the literal unsafe keyword and its machinery doesn't exist

The latter is how most ostensibly safe languages work. See Haskell's UnsafePerformIO, Swift's UnsafePointer, and Java's JNI for 3 examples off the top of my head.

The former is just a really gimped language that would have been a pain in the neck to implement libraries for (see other replies for examples).

Re: Bugs You'll Probably Only Have in Rust

#23

Post by Animats »

halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization.

There's a lot of code like that in the standard library. Somebody thought that safe code would be too slow, and used unsafe code for some local optimization. I tend to think this was overdone.

Here's a typical example, in crate "url": [1] unsafe conversion of a slice from a UTF-8 string by calling "str::from_utf8_unchecked". Minor speedup, but if the slice calculation is off, unsafe, because advancing through a bad (or misaligned slice of a?) UTF-8 string may cause a subscript error.

There's an effort underway to clean up some low-level Rust library crates, and "unsafe" appears rarely in ones that have been through that process. So there is forward progress. But few crates have been through that process yet.

(I've complained about this in the past, pointing to places There's a Rust fanboy who usually replies, without citing any references, denying everything. The usual excuses are 1) nobody really uses unsafe crate X much, 2) the crates fanboy is using contain less unsafe code than I'm finding.)

[1] https://github.com/servo/rust-url/blob/master/src/form_urlen...

Re: Bugs You'll Probably Only Have in Rust

#24

Post by bluejekyll »

I have to say, these RCA's of the various bugs are great for getting a better understanding of the internals of the language.

In a lot of ways it makes me trust Rust even more, because there is a deeper understanding of exactly how these guarantees are made.

Re: Bugs You'll Probably Only Have in Rust

#25

Post by jacquesm »

steveklabnik wrote: post #19
halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

I'll give you the shortest example: in order to build an operating system in Rust for x86, you need to do this: let p = 0xb8000 as *mut u8; VGA drivers use the memory mapped at 0xb8000 to drive the device. This creates a pointer, p, at that address. In order to demonstrate this is safe (okay so unsafe isn't in this example, creating p is safe, but writing to/reading from it is not), a language would have to know: 1.…

> That your code is running in kernel mode, that is the entire concept of ring 0 vs ring 3.

That need not be the case though. You could have a kernel side allocator that sets up the MMU to map that memory to a pointer that you return which lives in the space of the process. The MMU would take care of the required arithmetic to access the memory at its actual location using an offset.

That way you can map resources from real addresses into arbitrary addresses on the user side.

I think the correct term for this mechanism is 'system address translation'.

Re: Bugs You'll Probably Only Have in Rust

#26

Post by kbenson »

Animats wrote: post #23
halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization. There's a lot…

> (I've complained about this in the past, pointing to places There's a Rust fanboy who usually replies, without citing any references, denying everything. The usual excuses are 1) nobody really uses unsafe crate X much, 2) the crates fanboy is using contain less unsafe code than I'm finding.)

While I've actually seen you point this out multiple times here, and seen the response you describe, I can't help but feel you are characterizing it with unhelpful flair which is likely to result in nonconstructive responses.

Re: Bugs You'll Probably Only Have in Rust

#27

Post by [deleted] »

Animats wrote: post #23
halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization. There's a lot…

[deleted]

Re: Bugs You'll Probably Only Have in Rust

#28

Post by steveklabnik »

jacquesm wrote: post #25
steveklabnik wrote: post #19

Earlier quoted context omitted.

I'll give you the shortest example: in order to build an operating system in Rust for x86, you need to do this: let p = 0xb8000 as *mut u8; VGA drivers use the memory mapped at 0xb8000 to drive the device. This creates a pointer, p, at that address. In order to demonstrate this is safe (okay so unsafe isn't in this example, creating p is safe, but writing to/reading from it is not), a language would have to know: 1.…

> That your code is running in kernel mode, that is the entire concept of ring 0 vs ring 3. That need not be the case though. You could have a kernel side allocator that sets up the MMU to map that memory to a pointer that you return which lives in the space of the process. The MMU would take care of the required arithmetic to access the memory at its actual location using an offset. That way you can map resources fr…

The language would still have to understand all of that in order to write that kernel side allocator in safe code.

Re: Bugs You'll Probably Only Have in Rust

#29

Post by pcwalton »

Animats wrote: post #23
halestock wrote: post #17

Question for the rust folks - are there any features that wouldn't have been possible without "unsafe"? That is, if rust never had unsafe, would it have been fundamentally limited in any way? Or is it required for e.g. interoperability with C?

Outside of foreign functions, "unsafe" is used mostly for (sometimes premature) optimization. "Vec" needs "unsafe" because Rust can't talk about a partially initialized array. But "Vec" isn't very complicated. Maps are very complicated, and written as unsafe code. They could been written using "Vec" for storage, but they're not; they're at the raw storage level. That's a possibly premature optimization. There's a lot…

> Here's a typical example, in crate "url": [1] unsafe conversion of a slice from a UTF-8 string by calling "str::from_utf8_unchecked". Minor speedup, but if the slice calculation is off, unsafe, because advancing through a bad (or misaligned slice of a?) UTF-8 string may cause a subscript error.

Simon is a reasonable guy and is quite receptive to bug reports. If you think that that function should be made safe, feel free to file an issue on GitHub.

Note that URL parsing is quite performance-critical in browsers, especially where data: URLs are concerned. The code in question is used in Firefox.

Re: Bugs You'll Probably Only Have in Rust

#30

Post by jacquesm »

steveklabnik wrote: post #28
jacquesm wrote: post #25

Earlier quoted context omitted.

> That your code is running in kernel mode, that is the entire concept of ring 0 vs ring 3. That need not be the case though. You could have a kernel side allocator that sets up the MMU to map that memory to a pointer that you return which lives in the space of the process. The MMU would take care of the required arithmetic to access the memory at its actual location using an offset. That way you can map resources fr…

The language would still have to understand all of that in order to write that kernel side allocator in safe code.

I don't see how that follows. The language can't possibly understand the intricacies of what the MMU is capable of (besides, every MMU is different), and as far as the language is concerned what is returned is simply a valid offset and a length to go with it to indicate where the allocated segment ends.
Post reply on HN