Secure Rust Guidelines • Hacker News Board

Re: Secure Rust Guidelines

#51

Post by tialaramex » Sat, Mar 21, 2020, 8:11 PM UTC

rectang wrote: post #44

mbrubeck wrote: post #35

Earlier quoted context omitted.

Agreed. There are definitely good arguments for using abort-on-panic, and doing isolation and recovery at the process level rather than the thread level. This is what we do with all the Rust code in Firefox, for example. Unwind safety is a real issue. I have some personal experience with fixing panic-safety issues in unsafe Rust code: https://github.com/servo/rust-smallvec/pull/103 I wrote a bit more about it here: h…

Even isolation and recovery at the process level does not guarantee that a multi-process application will have no invalid state! By the time invalid state is detected in one process and a panic happens, the invalid state may have already propagated to another process via IPC messaging. And even taking down an entire multi-process application doesn't fully protect against invalid state, if that invalid state has wound…

Your goal should be for your software to behave correctly regardless of persistent state. That is, all persistent states are valid, or else that's a grave bug.

If the situation is "Program fails when restoring from state X" the priority for "Don't fail when restoring from state X" is higher than for "Don't cause whatever happened that results in state X".

Example: Let's say your code believes 'foo' is supposed to be a file with an XML structure in it. A user reports that something went wrong and now the program crashes, the file 'foo' is now exactly 4096 bytes (one page on most architectures) of binary noise.

Correct priority: #1 Make the program work when 'foo' is not an XML file. If possible (maybe 'foo' is storing animated profile pictures for a chat program) it should carry on, if that's impractical (maybe 'foo' defines which model of X-ray machine we're hooked up to, best not to press on without knowing) it should give a clear error explaining what's wrong.

#2 only after fixing #1 figure out how 'foo' gets corrupted and try to solve that.

Re: Secure Rust Guidelines

#52

Post by baby » Sat, Mar 21, 2020, 8:28 PM UTC

masklinn wrote: post #41

baby wrote: post #39

Earlier quoted context omitted.

Honestly all C programs that fail to allocate panic as well. What else can you do?

> Honestly all C programs that fail to allocate panic as well. I'd say that most C programs that fail to allocate panic (if only because e.g. they're running over-committing anyway so allocations never fail), but it's not true that all of them do. There are systems which do handle heap allocation failures "properly", usually ones which run on systems which are both resource-constrained enough that allocation failure…

I've looked at these types of systems and they would still crash, actually even if you want to gracefully handle failed allocations it is possible that your program will crash due to lack of memory

Re: Secure Rust Guidelines

#53

Post by baby » Sat, Mar 21, 2020, 8:30 PM UTC

Paul-ish wrote: post #46

baby wrote: post #36

Earlier quoted context omitted.

No，if you are writing secure code then a panic is a denial of service. Their point is that you should never panic, unless you do reach an unrecoverable error.

It seems like, in the absence of panics, there is a dichotomy between "You should know everything that can happen to your program and account for it." and "If you hit some unknown state, carry on." The first one is impossible, the second sounds dangerous. When you get into this dichotomy, panicking seems like a reasonable third option. We should strive for the first, but avoid the second. Nobody wants their code to p…

Yeah that's what is lacking from my comment: expect panics to happen and attempt to gracefully recover whenever that happens. This is not easy though.

Re: Secure Rust Guidelines

#54

Post by a1369209993 » Sat, Mar 21, 2020, 8:47 PM UTC

dathinab wrote: post #50

Paul-ish wrote: post #45

Earlier quoted context omitted.

I agree wholeheartedly. If we tell people "Don't write code that panics" then we collectively start writing code that assume panics don't happen. Suddenly panics become a footgun in Rust, and it isn't the safe language we want it to be. If I'm feeling in a mood, I think we should panic all the time. We should have a "panic monkey" tool that places panics at random parts of the code, to see how panics are handled. Eg…

I can only agree diesel had (maybe still has) a serve bug around connection reuse and panics. Panics in rust originally where a pretty nice way to have a error kennel pattern and as long as you always kill the whole error kennel in which the panic occurred and don't share mutable state between error kennels this fully save from any inconsistent state. Sure panics across FFI boundaries can't be done nice at all, but t…

You don't need unwind safety (for anything, ever); a error kennel is more commonly known as a "process"; when a panic occurs

> you always kill the whole [process] in which the panic occurred and don't share mutable state between [process]s

Processes are designed for this, including making it very hard to accidentaly share mutable state (using eg mmap(MAP_SHARED) explicitly).

Re: Secure Rust Guidelines

#55

Post by a1369209993 » Sat, Mar 21, 2020, 10:40 PM UTC

cesarb wrote: post #34

masklinn wrote: post #12

Earlier quoted context omitted.

Yes that is one of the primary failures of Rust at the moment: to my knowledge it currently has no good way to safely manage allocation failures (it also has serious issues with stack overflows). This is an issue with all heap-allocating construct, not just collections but also Box or Rc. > I wish Rust had some easy way to recover from out-of-memory situations when using the standard library. I have been considering…

The problem with allocation failure is that it's non-local. Suppose thread A does a huge allocation of several hundred megabytes. The best case scenario is that this allocation fails cleanly; the worst case scenario is that this allocation succeeds but causes a small 1024-byte allocation in an unrelated thread B to fail (and it doesn't have to be multiple threads, this can happen even within a single thread). I don't…

The horrible part is that this isn't even limited to threads; a competely separate process (even, eg, a browser JS engine running 'completely' untrusted remote code) can cause your program to hit a allocation failure, and then not have enough memory to preform nontrivial error-handling.

Re: Secure Rust Guidelines

#56

Post by a1369209993 » Sat, Mar 21, 2020, 11:02 PM UTC

tialaramex wrote: post #51

rectang wrote: post #44

Earlier quoted context omitted.

Even isolation and recovery at the process level does not guarantee that a multi-process application will have no invalid state! By the time invalid state is detected in one process and a panic happens, the invalid state may have already propagated to another process via IPC messaging. And even taking down an entire multi-process application doesn't fully protect against invalid state, if that invalid state has wound…

Your goal should be for your software to behave correctly regardless of persistent state. That is, all persistent states are valid, or else that's a grave bug. If the situation is "Program fails when restoring from state X" the priority for "Don't fail when restoring from state X" is higher than for "Don't cause whatever happened that results in state X". Example: Let's say your code believes 'foo' is supposed to be…

Nitpick: this isn't really about persistent state specifically, but any shared (and especially mutable) state, including but not limited to state shared across multiple temporally-nonoverlapping instances of the same program. Eg, compare when 'foo' is provided as input or fetched across a network.

Re: Secure Rust Guidelines

#57

Post by mbrubeck » Sun, Mar 22, 2020, 1:26 AM UTC

eeZah7Ux wrote: post #48

mbrubeck wrote: post #25

Earlier quoted context omitted.

Panicking is, in many ways, the best-case scenario for code that contains a bug. A bug that causes a panic is much easier to find through fuzzing, property testing, even static analysis or analyzing failures in production. It also prevents the bug from "infecting" other code by allowing the program to proceed in an invalid state. If you don't want a panic to take down the whole system, you can isolate the code in a t…

> Panicking is, in many ways, the best-case scenario for code that contains a bug. The very opposite: it's among the worst behaviors, short of creating a vulnerability. From any modern languages we should expect an exception to be raised.

"Panic" is essentially the Rust name for exceptions. They are even implemented the same as C++ exceptions in terms of codegen. The real differences are cultural rather than technical, as I touch on in this thread:

https://users.rust-lang.org/t/c-pitfalls-hard-to-avoid-that-...

Re: Secure Rust Guidelines

#58

Post by EvilTerran » Sun, Mar 22, 2020, 9:26 AM UTC

geofft wrote: post #31

EvilTerran wrote: post #20

Earlier quoted context omitted.

> Hoogle doesn't seem to find me a function like Rust's .get() in the standard library In practice, you don't really need one - the safe alternative to "xs !! n" is pattern-matching on the result of "drop n xs", as that's [] if xs has ≤n elements: https://www.haskell.org/onlinereport/standard-prelude.html#$...

Sure, that seems syntactically more cumbersome than 'if let Some(foo) = xs.get(n)' or 'xs.get(n).map(|foo| ...)' in Rust, but yes, you can do it. As I said, because both the Rust and Haskell versions are more cumbersome than using the version that raises an exception/panics, both Rust and Haskell's standard libraries choose to give you a syntactically-easier alternative that isn't a total function. All I'm saying is…

Sure, I don't disagree - I meant only to add context for anyone reading who was unfamiliar with Haskell, lest they come away with the impression that the lack of a .get()-equivalent was some kind of egregious oversight.

Re: Secure Rust Guidelines

#59

Post by eeZah7Ux » Sun, Mar 22, 2020, 2:47 PM UTC

mbrubeck wrote: post #57

eeZah7Ux wrote: post #48

Earlier quoted context omitted.

> Panicking is, in many ways, the best-case scenario for code that contains a bug. The very opposite: it's among the worst behaviors, short of creating a vulnerability. From any modern languages we should expect an exception to be raised.

"Panic" is essentially the Rust name for exceptions. They are even implemented the same as C++ exceptions in terms of codegen. The real differences are cultural rather than technical, as I touch on in this thread: https://users.rust-lang.org/t/c-pitfalls-hard-to-avoid-that-...

No, it does not have the same semantics of exceptions.

Re: Secure Rust Guidelines

#60

Post by dathinab » Sun, Mar 22, 2020, 7:11 PM UTC

a1369209993 wrote: post #54

dathinab wrote: post #50

Earlier quoted context omitted.

I can only agree diesel had (maybe still has) a serve bug around connection reuse and panics. Panics in rust originally where a pretty nice way to have a error kennel pattern and as long as you always kill the whole error kennel in which the panic occurred and don't share mutable state between error kennels this fully save from any inconsistent state. Sure panics across FFI boundaries can't be done nice at all, but t…

You don't need unwind safety (for anything, ever); a error kennel is more commonly known as a "process"; when a panic occurs > you always kill the whole [process] in which the panic occurred and don't share mutable state between [process]s Processes are designed for this, including making it very hard to accidentaly share mutable state (using eg mmap(MAP_SHARED) explicitly).

Sure, but having process error kernels isn't at all appropriate for many applications. E.g. you don't start a process for every http request (or actor request) your server handles (even starting a thread for everyone is considered to much overhead).

But not only is the overhead of "starting"/entering a error kernel a problem you also normally do share _immutable_ state (e.g. configs, lookup tables), as well as some _minor_ amount of mutable but well known to be panic safe state (e.g. simple performance counters and some caches). Which is now much harder to get right.

E.g. in a cache you want it potentially to be shared across error-kernels and re-use it after a kernel crashed but only if the panic didn't come from the cache. So the cache needs to be handled in it's own error-kernel which makes it important to make the intercommunication fast.

Sure you _can_ do all this with processes (by using both IPC and mmap together), but it's much harder to get right. Often this leads to either serve performance limitations or braking of the kernels surface (by using mmap in a bad way). It also tend to lead to code which is very dependent on the features of a specific operating system. Problems with managing startup/shutdown (no systemd isn't a solution ), moving resources between kernels etc.

So all in all processes make thinks more complex, less portable with very little to no benefits for many use cases. (Sandboxing of very large error-kernels like done in browsers is one point where it makes sense).