Live data from Hacker News

totally_safe_transmute, Line-by-Line (2021)

blog.yossarian.net

11–20 of 41 posts

Re: totally_safe_transmute, Line-by-Line (2021)

#11
post #6

This is cute, but I hope it never turns up in any real codebase! There’s an updated version with Windows support and better performance: https://github.com/John2143/totally-speedy-transmute/ What worries me is this macro, which “smuggles” the unsafe keyword past the forbid(unsafe_code) flag: https://github.com/John2143/totally-speedy-transmute/blob/ma... In my mind, this kind of capability makes Rust crate safety sca…

You can always smuggle unsafe past the compiler, it can't stop you even in principle. The totally safe transmute is at runtime, so an instrumented compiler cannot detect it (halting problem is in the way). You'd need runtime instrumentation of your binary. And even then, it's wildly impractical. If you let an application interact with the environment, tomorrow Linux or Windows could add a new magic file, or a special…

> You can always smuggle unsafe past the compiler, it can't stop you even in principle.

The linked updated library uses a different method: it literally smuggles the "unsafe" keyword past the safety checks by removing the space character from "un safe".

This can and should be caught by the compiler -- it has full access the syntax tree at every intermediate stage of compilation! Instead, the Cargo tool and the rustc compiler are simply keyword-searching for the literal string "unsafe", and are hence easily fooled.

Note that this updated method is not the same thing as the Linux process memory mapping and doesn't rely on OS APIs in any way. It is a purely compile-time hack, not a runtime one.

What I'd love to see is an healthy ecosystem of open-source packages that are truly safe: using a safe language, without any escape hatches, etc...

E.g.: Libraries for decoding new image formats like JPEG XL ought to be 100% safe, but they're not! This hampers adoption, because browsers won't import huge chunks of potentially dangerous code.

Now we can't have HDR photos on the web because of this specific issue: The Chromium team removed libjxl from the code base because of legitimate security concerns. A million other things are also going to be "unsupported" for a long time because of the perceived (and real!) dangers of importing third-party dependencies.

We'll be stuck with JPG, GIF, and PNG forever because there is no easy way to share code that is truly safe in the "pure function with no side-effects" sense.

PS: This type of issues is also the root-cause of issues like Log4J and various XML decoder security problems. By default, most languages allow arbitrary code even for libraries that ought to perform "pure transformations" of one data format to another, such as reaching out to LDAP servers over TCP/IP sockets, as in the case with Log4J. Similarly, XML decoders by default use HTTP to retrieve referenced files, which is madness. Even "modern" formats like YAML make this mistake, with external file references on by default!

> What you probably want is a runtime VM, like WASM.

Sure, that's one way to sandbox applications, but in principle it ought to be entirely possible to have a totally safe ahead-of-time compiled language and/or standard library.

Rust is really close to this goal, but falls just short because of tricks like this macro backdoor. (It would also need a safe subset of the standard library.)

Re: totally_safe_transmute, Line-by-Line (2021)

#14
post #7
post #4

Why don't the safe file I/O operations panic when /proc/self/mem is opened for writing? I understand why they don't want to make all of File I/O unsafe just for edge cases like this, but shouldn't this be handled at runtime?

It's hard to do by accident and is pretty unix specific. Also, what if you really want do access this and it panics?

> Also, what if you really want do access this and it panics?

That's the easy part, you'd provide some OS-specific API for getting this file handle, call it `std::os::fs::proc()` or thereabouts, and make it an `unsafe` function. The "hard to do" is the bigger problem, because if you're providing an unsafe alternative then you'd like to plug all the holes in the safe interface, which AFAIK is non-trivial.

Re: totally_safe_transmute, Line-by-Line (2021)

#16
post #4

Why don't the safe file I/O operations panic when /proc/self/mem is opened for writing? I understand why they don't want to make all of File I/O unsafe just for edge cases like this, but shouldn't this be handled at runtime?

Because you could just do the Rust equivalent of system("dd of=/proc/myprocess/mem ...") instead, so it would be security theater. Memory safety just isn't a part of the default Unix model.

Note the emphasis on "default" above; you can use the Linux sandboxing features such as seccomp-bpf to build a sandbox which is truly memory-safe, closing this hole. The OS is in charge of the features it exposes, and Rust can't do much about that.

Note also that the existence of totally_safe_transmute doesn't mean that Rust's memory safety features are pointless. Empirically, memory-safe programming languages result in far fewer memory safety vulnerabilities, because they make exploitation way harder.

Re: totally_safe_transmute, Line-by-Line (2021)

#17
post #6

Earlier quoted context omitted.

You can always smuggle unsafe past the compiler, it can't stop you even in principle. The totally safe transmute is at runtime, so an instrumented compiler cannot detect it (halting problem is in the way). You'd need runtime instrumentation of your binary. And even then, it's wildly impractical. If you let an application interact with the environment, tomorrow Linux or Windows could add a new magic file, or a special…

> You can always smuggle unsafe past the compiler, it can't stop you even in principle. The linked updated library uses a different method: it literally smuggles the "unsafe" keyword past the safety checks by removing the space character from "un safe". This can and should be caught by the compiler -- it has full access the syntax tree at every intermediate stage of compilation! Instead, the Cargo tool and the rustc…

> The linked updated library uses a different method: it literally smuggles the "unsafe" keyword past the safety checks by removing the space character from "un safe".

> This can and should be caught by the compiler -- it has full access the syntax tree at every intermediate stage of compilation! Instead, the Cargo tool and the rustc compiler are simply keyword-searching for the literal string "unsafe", and are hence easily fooled.

This is not what's going on. No Rust tool performs any such string scanning. The "un safe" with the space is purely for aesthetic effect. In fact, the proc macro works exactly the same when modified to just use "unsafe" directly, and cargo-geiger still doesn't report any issues.

The real effect is that macros and proc macros from foreign crates are allowed to emit unsafe { ... } blocks in their output, even if the caller of the macro uses #![forbid(unsafe_code)]. Effectively, the output is considered as coming from "different crate" from the caller with the #![forbid], so it's not effected by the forbidden lint. This behavior was implemented in [0], which silences all lints (including forbidden lints) within the output of macros defined in foreign crates, except for certain lints which explicitly opt-in.

The #![forbid(unsafe_code)] inside the src/lib.rs defining the proc macro similarly doesn't do anything, since it restricts the definition of the proc macro, but not the output of the proc macro.

As far as I know, there is currently no way to deny unsafe code in the output of proc macros.

[0] https://github.com/rust-lang/rust/pull/52467

Re: totally_safe_transmute, Line-by-Line (2021)

#18
post #12

This is a really weird hack to say the least. More like a flex showing that the author can implement transmute without unsafe than something you’d really use.

The point is likely to show that Rust's "safety" is not absolute and it's possible to do lots of silly stuff with "safe" code.
Post reply on HN