Unsafe Zig Is Safer Than Unsafe Rust • Hacker News Board

Re: Unsafe Zig Is Safer Than Unsafe Rust

#51

Post by klodolph » Thu, Jan 25, 2018, 12:34 AM UTC

dbaupp wrote: post #46

klodolph wrote: post #41

Earlier quoted context omitted.

> This comment suggests you don't have much domain knowledge about how `unsafe` in Rust works, so I'm surprised you speak with such confidence. I hate being tone police, but jeez, we're having a discussion about Rust here and talking about my personal competency is inappropriate and unwelcome. The problem I'm talking about happens when you write libraries that contain "unsafe" blocks. You want to prove (or at least a…

> One known trap is that it is not sufficient to demonstrate that Rust code without "unsafe" blocks cannot observe unsafe behavior in your library. I'm curious: what does this mean/could you point me to the part of the paper that describes it? (Unfortunately, I don't have time to read all 34 pages at the moment.)

p. 66:3, the two paragraphs starting with "However, there is cause for concern..."

Re: Unsafe Zig Is Safer Than Unsafe Rust

#52

Post by klodolph » Thu, Jan 25, 2018, 12:45 AM UTC

tatterdemalion wrote: post #44

klodolph wrote: post #41

Earlier quoted context omitted.

> This comment suggests you don't have much domain knowledge about how `unsafe` in Rust works, so I'm surprised you speak with such confidence. I hate being tone police, but jeez, we're having a discussion about Rust here and talking about my personal competency is inappropriate and unwelcome. The problem I'm talking about happens when you write libraries that contain "unsafe" blocks. You want to prove (or at least a…

Proving the correctness of unsafe code is totally different from what you talked about, which was composing different abstractions with unsafe internals together. Users of safe Rust do not need to worry about whether the composition of two safe interfaces that use unsafe internally is safe unless one of those interfaces is incorrect. Your comment would suggest that users need to think about the untyped invariants of…

The problem with talking about this subject is that "safe" and "unsafe" are overloaded terms in Rust, so I can understand why you think I was talking about something different.

Let R be arbitrary Rust code with no "unsafe" blocks. Let X and Y be libraries with "unsafe" blocks. You can prove that R + X is safe, and prove that R + Y is safe, but you haven't yet proven R + X + Y is safe. This is the hard part, because without an understanding of what property of X and Y individually makes R + X + Y + Z + ... safe, we don't have a good definition for what makes an interface "safe".

And this is what I mean when I say that this is not only a pedagogical problem.

Re: Unsafe Zig Is Safer Than Unsafe Rust

#53

Post by stouset » Thu, Jan 25, 2018, 12:46 AM UTC

Retra wrote: post #47

adwhit wrote: post #27

Earlier quoted context omitted.

'unsafe' doesn't mean unsafe. Unsafe means "I can't convince the compiler that this is safe. But in my context, it is." If there is any way in which a function containing an `unsafe` block may be used unsafely (specifically, violating memory-safety), then that function must also be marked as unsafe.

That's what it means if you use it properly. If you write bad code, it means "this code will break everything and the compiler won't protect you." An `unsafe` block does nothing to guarantee that you're doing something safe, which is what you seem to be saying, even if it's not what you mean to say.

This is pretty much tautological, and nobody's arguing this point. However, you have the benefit of being able to narrow your search scope to the parts of your code marked `unsafe` instead of the entire project.

Most things don't need unsafe code. For the things that do, you must yourself uphold the invariant that all requirements of safety are being obeyed when transitioning out of an unsafe block. If you don't do this, bad things can happen. Other languages don't have this because they either don't offer Rust's safety guarantees in the first place, or the only way to circumvent them is to write code in C.

Re: Unsafe Zig Is Safer Than Unsafe Rust

#54

Post by dbaupp » Thu, Jan 25, 2018, 12:49 AM UTC

klodolph wrote: post #51

dbaupp wrote: post #46

Earlier quoted context omitted.

> One known trap is that it is not sufficient to demonstrate that Rust code without "unsafe" blocks cannot observe unsafe behavior in your library. I'm curious: what does this mean/could you point me to the part of the paper that describes it? (Unfortunately, I don't have time to read all 34 pages at the moment.)

p. 66:3, the two paragraphs starting with "However, there is cause for concern..."

Thanks!

I'm not convinced that the statement in the paper translates into what you said: the key piece of that paragraph is "or seems to be". The Leakpocalypse problem was one piece of code (crossbeam's scoped threads API) was relying on an invariant that doesn't actually hold ("destructors will always run"). It was, fundamentally, a bug in the `unsafe` code in crossbeam, meaning it was incorrect for crossbeam to call its API safe: the fact that it took multiple libraries to trigger in that case means nothing, it just happens to be the circumstances under which the problem was noticed.

Of course, to be fair, no-one had thought about this destructor property before, just implicitly relied on it, and so it does demonstrate the necessity for better understanding of/tools for unsafe code, which is what projects like RustBelt are pushing towards.

To summarise, I still don't see how these two sentences are different:

> no unsafe behavior is observable by clients of the library

> [clients] without "unsafe" blocks cannot observe unsafe behavior in [the] library

Indeed, I don't think it makes sense to even attempt to prove that clients with unsafe code can't observe unsafe behaviour (which seems to be the only way for the second sentence to differ from the first). The typical framing is that the safe code can be arbitrarily bad and there'll still be no undefined behaviour, but arbitrary `unsafe` can do anything, including writing directly to another library's data structures, which of course can easily cause UB (e.g. replace a Vec's data pointer with a null one).

Re: Unsafe Zig Is Safer Than Unsafe Rust

#55

Post by bjourne » Thu, Jan 25, 2018, 12:55 AM UTC

dbaupp wrote: post #50

bjourne wrote: post #48

Earlier quoted context omitted.

On 64bit Linux, stack frames are always aligned at 16 byte boundaries. The first 8 bytes of the frame contains the return address then there are 8 bytes of padding and then comes the stack allocations. I think the example is poorly constructed, because it is inconceivable that the address to the start of an array would not be aligned sizeof(int*) bytes.

The example is illustrative enough: all the array needs to be misaligned in practice is a small value on the stack near it, e.g. if the Rust code has `let x: u8 = 1;` inserted after the array (or, I imagine, `uint8_t x = 1;` in the C, etc.), then the array's address is odd.

Why would it be? I tried it with msvc and it always manages to put arrays in stack frames at 8 byte boundaries. I can't see why a compiler would not do that.

Re: Unsafe Zig Is Safer Than Unsafe Rust

#56

Post by stouset » Thu, Jan 25, 2018, 12:56 AM UTC

klodolph wrote: post #52

tatterdemalion wrote: post #44

Earlier quoted context omitted.

Proving the correctness of unsafe code is totally different from what you talked about, which was composing different abstractions with unsafe internals together. Users of safe Rust do not need to worry about whether the composition of two safe interfaces that use unsafe internally is safe unless one of those interfaces is incorrect. Your comment would suggest that users need to think about the untyped invariants of…

The problem with talking about this subject is that "safe" and "unsafe" are overloaded terms in Rust, so I can understand why you think I was talking about something different. Let R be arbitrary Rust code with no "unsafe" blocks. Let X and Y be libraries with "unsafe" blocks. You can prove that R + X is safe, and prove that R + Y is safe, but you haven't yet proven R + X + Y is safe. This is the hard part, because w…

You have restated your position, but it is still incorrect in the context of this discussion. Even your original statement of "R be[ing] arbitrary Rust code with no 'unsafe' blocks" is problematic: any Rust code is, very unavoidably, built upon a foundation of unsafe code. It has to be, because it's running on an "unsafe" processor. And yet, any safe Rust code in the core library (barring a safeness bug) is obviously safely composable with any other safe Rust code precisely because it obeys safety guarantees when transitioning from unsafe to safe. The fact that you can mistakenly conceive of Rust code that somehow avoids any internal unsafety simply reinforces how obvious this simple fact is.

But using your original problem statement, if R is safe and X and Y use unsafe code but do not expose any unsafe interfaces, then either R + X + Y is safe or one of [X, Y] has a safety bug and is inaccurately marking an unsafe interface as safe.

This is a generally unsolvable problem, and every other language has this problem as well; the difference being that in most other languages you're typically forced to write the unsafe code in C (where one has much greater variety of footguns available at their disposal). If I write a Ruby FFI wrapper for buggy C code whose interfaces bleed "unsafe" (from the perspective of the Ruby VM) behavior, then I am liable to experience crashes and memory corruption bugs. The only difference here is that Rust allows you to break the seal on the warranty without switching to a different language.

Re: Unsafe Zig Is Safer Than Unsafe Rust

#57

Post by devit » Thu, Jan 25, 2018, 1:03 AM UTC

I think both "as" and using "transmute" for non-exceptional circumstances are mistakes in Rust.

There should instead be a bunch of type-specific cast operators that can check things like alignment and that what you intended to be a zero-extending integer cast is not in fact truncating to a smaller integer type, and so on.

It's not too late to deprecate "as" and discourage using "transmute" in favor of those.

Re: Unsafe Zig Is Safer Than Unsafe Rust

#58

Post by [deleted] » Thu, Jan 25, 2018, 1:18 AM UTC

stochastic_monk wrote: post #14

kzrdude wrote: post #3

Rust is a language that offers you lots of compile time checks, and an escape hatch called unsafe that says “trust the programmer here.” Yes, it is possible—and easy—to make mistakes in the place where you have asked to be trusted, not checked. We have a big pedagogical task ahead of us in teaching safe practices for unsafe Rust, and defensive coding practices in unsafe Rust. We should also think of if we can improve…

C++ largely suffers from the same problems. Often a C++ programmer can write code which relies on iterators and containers which is quite safe and difficult to mess up, while for a variety of highly-specialized applications, mixtures of packed structs, pointer arithmetic, and arbitary sequences of binary data need to be handled with utmost care. Knowing when to use which set of tools and how to safely glue them toget…

[deleted]

Re: Unsafe Zig Is Safer Than Unsafe Rust

#59

Post by Arnavion » Thu, Jan 25, 2018, 1:22 AM UTC

Animats wrote: post #40

kevingadd wrote: post #34

Earlier quoted context omitted.

C99 has this, I believe? https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html

That's for local variables. Microsoft and Linus Torvalds didn't like it, because it's a way to suddenly cause unexpected stack growth of arbitrary size. That feature was made optional in C++11, and Microsoft never implemented it.

FWIW Microsoft does have SAL annotations to do the same thing. For example fread's prototype is

    size_t fread(
        _Out_writes_bytes_(_ElementSize*_Count) void * _DstBuf,
        _In_ size_t _ElementSize,
        _In_ size_t _Count,
        _Inout_ FILE * _File
    );

https://docs.microsoft.com/en-us/visualstudio/code-quality/a...

Re: Unsafe Zig Is Safer Than Unsafe Rust

#60

Post by devit » Thu, Jan 25, 2018, 1:23 AM UTC

stochastic_monk wrote: post #14

kzrdude wrote: post #3

Rust is a language that offers you lots of compile time checks, and an escape hatch called unsafe that says “trust the programmer here.” Yes, it is possible—and easy—to make mistakes in the place where you have asked to be trusted, not checked. We have a big pedagogical task ahead of us in teaching safe practices for unsafe Rust, and defensive coding practices in unsafe Rust. We should also think of if we can improve…

C++ largely suffers from the same problems. Often a C++ programmer can write code which relies on iterators and containers which is quite safe and difficult to mess up, while for a variety of highly-specialized applications, mixtures of packed structs, pointer arithmetic, and arbitary sequences of binary data need to be handled with utmost care. Knowing when to use which set of tools and how to safely glue them toget…

Container and iterator code is not safe at all since there is no bounds checking by default and no protection against iterator invalidation, which can both cause writes to memory outside the intended object and thus a catastrophic outcome.

There is no safe subset of C/C++ unless you just don't use pointers or references at all (and refrain from using any library that is not safe which includes large parts of the standard library like all the containers), or you write it in Rust or an equivalent language with lifetimes and linear types and automatically translate it to C/C++ somehow.