Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No-Panic Rust: A Nice Technique for Systems Programming

blog.reverberate.org

Post reply on HN
Print view View original

111–120 of 143 posts

Re: No-Panic Rust: A Nice Technique for Systems Programming

#111

Post by ninkendo »

exDM69 wrote: post #104
ninkendo wrote: post #102

Earlier quoted context omitted.

All the things you describe are done automatically on program exit, even if the program is SIGKILL’ed. The kernel cleans up file descriptors, database transactions are rolled back automatically when the client disconnects (which it should be observed to be when the program exits and the kernel closes the connection as part of cleanup), and I’m not sure what you mean about mutexes, but if you mean in-memory ones, thos…

I'm well aware of this and good that the option exists to bail out with abort instead. But there are still cases where you would like to fsync your mmaps, print out a warning message or just make sure your #[should_panic] negative tests don't trigger false positives in your tooling (like leak detectors or GPU validators) or abort the whole test run. It's not perfect by any means but it's better than potentially corru…

Right, sorry for the patronizing tone, I’m sure you know all this.

But I tend to lament the overall tendency for people to write cleanup code in general for this kind of thing. It’s one of those “lies programmers believe about X” kinds of scenarios. Your program will crash, and you will hit situations where your cleanup code will not run. You could get OOM killed. The user can force quit you. Hell, the power could go out! (Or the battery could go dead, etc.)

Nobody should ever write code that is only correct if they are given the opportunity to perfectly clean up after any failure that happens.

I see this all the time: CLI apps that trap Ctrl-C and tell you you can’t quit (yes I bloody well can, kill -9 is a thing), apps which don’t bother double checking that the files they left behind on a previous invocation are actually still used (stale pid files!!!), coworkers writing gobs of cleanup code that could have been avoided by simply doing nothing, etc etc.

Re: No-Panic Rust: A Nice Technique for Systems Programming

#112

Post by nicce »

dathinab wrote: post #108
nicce wrote: post #103

Earlier quoted context omitted.

I would say that you are using them incorrectly if you assume them as recoverable. You should make everything you can so that they never happen. However, since it is still possible to have them in a place where the exiting the process is not okay, it was beneficial to add a way to recover from them. It does not mean that they are designed to be recoverable. > this is where most of it's overhead comes from Overhead co…

> I would say that you are using them incorrectly if you assume them as recoverable. no it's them being recoverable at well defined boundaries is a _fundamental_ design aspect of rust > Overhead comes from the cleaning process. If you don't clean properly, you might leak information or allocate more resources than you should. and that same cleanup process makes it recoverable

Even the book uses the word "unrecoverable". The wording is communication. The intention is not to recover them, while it could be possible.

https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-...

Re: No-Panic Rust: A Nice Technique for Systems Programming

#114

Post by swiftcoder »

gpm wrote: post #42
andyferris wrote: post #9

This seems to obviate a lot of Rust's advantages (like a good std library). I wonder what it would take to write a nopanic-std library? Panics really seem bad for composability. And relying on the optimzer here seems like a fragile approach. (And how is there no -nopanic compiler flag?)

Rust doesn't want to add any proof-system that isn't 100% repeatable, reliable, and forwards compatible to the language. The borrow checker is ok, because it meets those requirements. The optimizer based "no panic" proof system is not. It will break between releases as LLVM optimizations change, and there's no way to avoid it. Trying to enforce no-panics without a proof system helping out is just not a very practical…

> This code is obviously correct. It never panics

It doesn't panic within the code you typed, but it absolutely still can panic on OOM. Which is sort of the problem with "no panic"-style code in any language - you start hitting fundamental constructs that can can't be treated as infallible.

> Basically every practical language has some form of "this should never happen" root.

99% of "unrecoverable failures" like this, in pretty much every language, are because we treat memory allocation as infallible when it actually isn't. It feels like there is room in the language design space for one that treats allocation as a first-class construct, with suitable error-handling behaviour...

Re: No-Panic Rust: A Nice Technique for Systems Programming

#115

Post by swiftcoder »

wongarsu wrote: post #19
andyferris wrote: post #9

This seems to obviate a lot of Rust's advantages (like a good std library). I wonder what it would take to write a nopanic-std library? Panics really seem bad for composability. And relying on the optimzer here seems like a fragile approach. (And how is there no -nopanic compiler flag?)

The standard library is slowly adding non-panicking options. The article shows some of them (like vec.push_within_capacity()) and ignores some others (vec.get_unchecked()). There is still a lot of work to do, but it is an area where a lot of work gets done. The issue is just that a) Rust is still a fairly young language, barely a decade old counting from 1.0 release, and b) Rust is really slow and methodical in addin…

To be slightly pedantic, Vec::get() is the non-panicking version. Vec::get_unchecked() is just the version thereof that elides the bounds check.

Re: No-Panic Rust: A Nice Technique for Systems Programming

#116

Post by jerf »

pedromsrocha wrote: post #84

This blog post is very interesting, using Rust’s compiler optimizer as a theorem prover. This makes me wonder: are there any formal specifications on the complexity of this "optimizer as theorem prover"? Specifically, how does it handle recursion? Consider, for example, the following function, which decrements a number until it reaches zero. At each step, it asserts that the number is nonzero before recursing: fn rec…

> This makes me wonder: are there any formal specifications on the complexity of this "optimizer as theorem prover"?

Basically, the promise here "We formally promise not to promise anything other than the fact that optimized code should have 'broadly' the same effects and outputs as non-optimized code, and if you want to dig into exactly what 'broadly' means prepare to spend a lot of time on it". Not only are there no promises about complexity, there's no promises that it will work the same on the same code in later versions, nor that any given optimization will continue firing the same way as you add code.

You can program this way. Another semi-common example is taking something like Javascript code and carefully twiddling with it such that a particular JIT will optimize it in a particular way, or if you're some combination of insane and lucky, multiple JITs (including multiple versions) will do some critical optimization. But it's the sort of programming I try very, very hard to avoid. It is a path of pain to depend on programming like this and there better be some darned good reason to start down that path which will, yes, forever dominate that code's destiny.

Re: No-Panic Rust: A Nice Technique for Systems Programming

#117

Post by vlovich123 »

dathinab wrote: post #101

> fn check_invariant(&self) { > unsafe { assert_unchecked(self.ofs self.data.len()) } > } Is fundamentally unsound `check_invariant` needs to be unsafe as it doesn't actually check the invariant but tells the compiler to blindly assume they hold. Should probably also be named `assume_invariant_holds()` instead of `check_invariant()`.

I personally would have used the assume crate but I guess this got standardized more recently. They call out the safeness requirement and that it’s a sharp edge but like you I think they understate the sharpness and danger.

Re: No-Panic Rust: A Nice Technique for Systems Programming

#118

Post by vlovich123 »

wongarsu wrote: post #40
haberman wrote: post #36

Earlier quoted context omitted.

While I would love this to be true, I'm not sure that this design can statically prove anything. For an assert to fail, you would have to actually execute a code sequence that causes the invariant to be violated. I don't see how the compiler could prove at compile time that the invariants are upheld.

An assert is just a fancy `if condition { panic(message) }`. If the optimizer can show that condition is always false, the panic is declared as dead code and eliminated. The post uses that to get the compiler to remove all panics to reduce the binary size. But you can also just check if panic code was generated (or associated code linked in), and if it was then the optimizer wasn't able to show that your assert can't…

To be clear as there’s a lot of nuance. Assert unchecked is telling the compiler the condition must always hold. The optimizer and compiler don’t make any assumption about the assert. That information is then used by the compiler to optimize away checks it otherwise would have to do (eg making sure an Option is Some if you call unwrap).

If you have an assumption that gives unhelpful information, the optimizer will emit panic code. Worse, if the assumption is incorrect, then the compiler can easily miscompile the code (both in terms of UB because of an incorrectly omitted panic path AND because it can miscompile surprising deductions you didn’t think of that your assumption enables).

I would use the assume crate for this before this got standardized but very carefully in carefully profiled hotspots. Wrapping it in a safe call as in this article would have been unthinkable - the unsafe needs to live exactly where you are making the assumption, there’s no safety provided by the wrapper. Indeed I see this a lot where the safety is spuriously added at the function call boundary instead of making the safety the responsibility of the caller when your function wrapper doesn’t actually guarantee any of the safety invariants hold.

Re: No-Panic Rust: A Nice Technique for Systems Programming

#119

Post by vlovich123 »

haberman wrote: post #47
wongarsu wrote: post #40

Earlier quoted context omitted.

An assert is just a fancy `if condition { panic(message) }`. If the optimizer can show that condition is always false, the panic is declared as dead code and eliminated. The post uses that to get the compiler to remove all panics to reduce the binary size. But you can also just check if panic code was generated (or associated code linked in), and if it was then the optimizer wasn't able to show that your assert can't…

Ah, I see what you are saying. Yes, if the optimizer is able to eliminate the postcondition check, I agree that it would constitute a proof that the code upholds the invariant. The big question is how much real-world code the optimizer would be capable of "solving" in this way. I wonder if most algorithms would eventually be solvable if you keep breaking them down into smaller pieces. Or if some would have some step…

To be clear, the optimizer doesn’t uphold the post condition. It just says “the post condition is usable to elide other checks”. But if the condition itself is incorrect, the compiler will STILL elide those checks. If the compiler could prove the condition on its own it would have done so. That’s why that assert is named unchecked and is itself unsafe!

Re: No-Panic Rust: A Nice Technique for Systems Programming

#120

Post by vlovich123 »

Yoric wrote: post #99
haberman wrote: post #47

Earlier quoted context omitted.

Ah, I see what you are saying. Yes, if the optimizer is able to eliminate the postcondition check, I agree that it would constitute a proof that the code upholds the invariant. The big question is how much real-world code the optimizer would be capable of "solving" in this way. I wonder if most algorithms would eventually be solvable if you keep breaking them down into smaller pieces. Or if some would have some step…

I wonder if there is a simple way to stop compilation quickly, with a readable error message, if the optimizer isn't able to eliminate the check. edit: There is https://github.com/dtolnay/no-panic

> Functions that require some amount of optimization to prove that they do not panic may no longer compile in debug mode after being marked #[no_panic].

So you’re probably going to have to protect this with a cfg_attr to only apply the no_panic in release only.

Post reply on HN