Concurrency in Rust • Hacker News Board

Re: Concurrency in Rust

#81

Post by Manishearth » Sun, Mar 27, 2016, 6:27 PM UTC

quotemstr wrote: post #75

Manishearth wrote: post #13

Earlier quoted context omitted.

A failed malloc aborts the process. If this is important to you, don't use the heap abstractions in the stdlib then. This is no different from the situation in C++. (You can also plug in a custom allocator which behaves differently)

No custom allocator can make a task that fails to allocate gracefully report an error. Rust's error handling design is just terrible, and mostly a consequence of eschewing exceptions. Had Rust opted for exceptions, it'd be a much better, and actually usable, language. Rust's terribly error-handling strategy is the chief reason not to use it.

> and mostly a consequence of eschewing exceptions.

A panic is the same thing as an exception. If you want to catch a panic, use recover(), it's meant to be used exactly for these end-of-the-world panic scenarios (and for FFI/etc).

You can plug in a custom allocator that panics on OOM (I think the standard one aborts).

As Steve mentioned, custom allocators can mean two things. The type that exists in rust today is one where you can make OOM panic, but not have allocation methods return Result. A planned extension will let you have allocators with different semantics entirely work with stdlib types (via defaulted type parameters); and this will let you use regular error handling with stdlib types too.

Re: Concurrency in Rust

#82

Post by quotemstr » Sun, Mar 27, 2016, 6:30 PM UTC

Manishearth wrote: post #80

quotemstr wrote: post #76

Earlier quoted context omitted.

Bullshit. Every part of the C++ standard library (which is _much_ bigger than Rust's) can gracefully propagate resource exhaustion errors, including memory allocation failure, to callers. That Rust can't is a design flaw. > there are situations where your application will just abort randomly That's sure as hell not the case in any environment I choose to use.

> can gracefully propagate resource exhaustion errors, including memory allocation failure, to callers only on malloc. If the kernel overcommits, your process will abort when you try to use the memory, possibly way after the malloc and there's nothing you can do about it. That's the point being made here. > That Rust can't is a design flaw. (This is false, see Steve's reply above about this)

The world is not Linux. I happen to believe that believe that overcommit in the Linux kernel is a disgrace. It is, however, at least possible to disable it. It's not possible to retroactively add real exceptions to Rust, or to change the signature of all memory-allocation function to return Result.

Rust is supposed to be a general-purpose systems programming language, not a Linux programming language. Windows does not overcommit. A correctly configured Linux system does not overcommit. Lost of embedded systems don't (and can't) overcommit. Are you saying all of these people should avoid Rust's standard library?

> > That Rust can't is a design flaw.

> (This is false, see Steve's reply above about this)

It's clear that my opinion differs from that of many Rust developers and users. I still think I'm correct, that these developers and users are misguided, and that as Rust attempts to fill more niches, experience will show that my position is the correct one.

All I can say is that I personally will not use any language that bakes cornucopian assumptions about memory baked into its core library. I know that you say that it's possible to just avoid stdlib --- but the temptation to use it will be irresistible, and once somebody succumbs to the temptation, the entire program is now capable of aborting irrecoverably.

I will stick with languages . Modern C++ is safe and expressive enough, and it correctly reacts to resource exhaustion.

Re: Concurrency in Rust

#83

Post by steveklabnik » Sun, Mar 27, 2016, 6:43 PM UTC

quotemstr wrote: post #79

steveklabnik wrote: post #77

Earlier quoted context omitted.

> No custom allocator can make the task gracefully report failure > instead of panicing. So, first of all, "custom allocators" means two things: * overloading the allocator that's used by liballoc, and the crates that depend on it, like libstd * other allocators entirely The first is described here: https://doc.rust-lang.org/book/custom-allocators.html And the second is still in RFCs: https://github.com/rust-lang/rfc…

> why is panic-ing bad, but an exception good? Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts. Even if abort-on-panic were to be killed as a legal mode of operation, and even if the stigma were to be removed from std::panic::recover, we'd still be left with a language with two error handling st…

This is an area where you just can't actually please everyone. I have heard the same opinions you've expressed in this thread, just as strongly, for even including unwinding at all. That aborts should be the only option, and that the cost of unwinding is far too high to be included in a true systems language.

Language design is tough. I'm glad we have multiple languages.

Re: Concurrency in Rust

#84

Post by Manishearth » Sun, Mar 27, 2016, 6:48 PM UTC

quotemstr wrote: post #82

Manishearth wrote: post #80

Earlier quoted context omitted.

> can gracefully propagate resource exhaustion errors, including memory allocation failure, to callers only on malloc. If the kernel overcommits, your process will abort when you try to use the memory, possibly way after the malloc and there's nothing you can do about it. That's the point being made here. > That Rust can't is a design flaw. (This is false, see Steve's reply above about this)

The world is not Linux. I happen to believe that believe that overcommit in the Linux kernel is a disgrace. It is, however, at least possible to disable it. It's not possible to retroactively add real exceptions to Rust, or to change the signature of all memory-allocation function to return Result. Rust is supposed to be a general-purpose systems programming language, not a Linux programming language. Windows does no…

> The world is not Linux.

Sure, but if linux has this issue, then C++ programs on linux will also have this issue, and the language can't solve that. That's all my point was.

> or to change the signature of all memory-allocation function to return Result.

When custom allocators part 2 happens, you can. I've already argued the "real exceptions" part above.

> Rust is supposed to be a general-purpose systems programming language, not a Linux programming language. Windows does not overcommit. A correctly configured Linux system does not overcommit. Lost of embedded systems don't (and can't) overcommit. Are you saying all of these people should avoid Rust's standard library?

No. My point was simply that no language has a complete solution to this problem.

Most people don't need to worry about OOM; abort-on-OOM is the expected behavior. For the people who do, there is a mechanism to handle it, as explained above. I can't help it if you have an idealogical issue with that mechanism. But ultimately, it works and can be used.

Re: Concurrency in Rust

#85

Post by Manishearth » Sun, Mar 27, 2016, 6:48 PM UTC

quotemstr wrote: post #79

steveklabnik wrote: post #77

Earlier quoted context omitted.

> No custom allocator can make the task gracefully report failure > instead of panicing. So, first of all, "custom allocators" means two things: * overloading the allocator that's used by liballoc, and the crates that depend on it, like libstd * other allocators entirely The first is described here: https://doc.rust-lang.org/book/custom-allocators.html And the second is still in RFCs: https://github.com/rust-lang/rfc…

> why is panic-ing bad, but an exception good? Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts. Even if abort-on-panic were to be killed as a legal mode of operation, and even if the stigma were to be removed from std::panic::recover, we'd still be left with a language with two error handling st…

> Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts.

recover() exists. You're right, there's a stigma to it, because you're not supposed to use it unless you really need to (hence, no programmer confusion). It's supposed to be used for situations like:

- Catching panics before crossing an FFI boundary

- End-of-the-world situations like OOM where you want to still handle it somehow

- Ensuring that applications can recover from internal panics in libraries (though there should be little to no panics in the libraries anyway)

The stigma for recover is for using it where you're not supposed to; as a substitute for regular error handling. In this situation, you are supposed to, so the stigma doesn't apply.

The fact that it's not a first-class primitive seems mostly irrelevant to me. Rust does a lot of things in library functions and types, even our concurrency safety mechanisms are something that can be duplicated in a library. As long as it can be used, what does it matter?

The fact that you can set the panic handler at runtime is also irrelevant. If you want to catch panics, don't do that.

Re: Concurrency in Rust

#86

Post by quotemstr » Sun, Mar 27, 2016, 6:58 PM UTC

Manishearth wrote: post #84

quotemstr wrote: post #82

Earlier quoted context omitted.

The world is not Linux. I happen to believe that believe that overcommit in the Linux kernel is a disgrace. It is, however, at least possible to disable it. It's not possible to retroactively add real exceptions to Rust, or to change the signature of all memory-allocation function to return Result. Rust is supposed to be a general-purpose systems programming language, not a Linux programming language. Windows does no…

> The world is not Linux. Sure, but if linux has this issue, then C++ programs on linux will also have this issue, and the language can't solve that. That's all my point was. > or to change the signature of all memory-allocation function to return Result. When custom allocators part 2 happens, you can. I've already argued the "real exceptions" part above. > Rust is supposed to be a general-purpose systems programming…

> Sure, but if linux has this issue, then C++ programs on linux will also have this issue

Why even bring Linux into the discussion? A Rust program running on Windows has the same problem.

> No. My point was simply that no language has a complete solution to this problem.

A correct C++ program running on Windows will not spuriously abort. Neither will a C++ program running on a Linux system configured not to overcommit. That some Linux systems can be configured to kill processes at arbitrary times is not an excuse for Rust to be sloppy with memory allocation.

C++ and many other languages do, in fact, have complete solutions to this issue, and that Rust does not is a serious deficiency, one serious enough to prompt me to prefer other languages despite Rust's other advantages.

Re: Concurrency in Rust

#87

Post by Rusky » Sun, Mar 27, 2016, 7:02 PM UTC

quotemstr wrote: post #79

steveklabnik wrote: post #77

Earlier quoted context omitted.

> No custom allocator can make the task gracefully report failure > instead of panicing. So, first of all, "custom allocators" means two things: * overloading the allocator that's used by liballoc, and the crates that depend on it, like libstd * other allocators entirely The first is described here: https://doc.rust-lang.org/book/custom-allocators.html And the second is still in RFCs: https://github.com/rust-lang/rfc…

> why is panic-ing bad, but an exception good? Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts. Even if abort-on-panic were to be killed as a legal mode of operation, and even if the stigma were to be removed from std::panic::recover, we'd still be left with a language with two error handling st…

Exceptions can be turned into aborts in C++ as well, and are in many types of programs, because exceptions do have downsides for some problem domains. If Rust forced exceptions on everyone, there'd be people complaining about that just like you're complaining now.

I see the split between `Result` and `panic!` as more like Java's split between checked and runtime exceptions, except `Result` is much more usable than checked exceptions because it's part of the main data flow path, and so can use method chaining combinators instead of unwieldy try/catch blocks. OOM in Java is, like in Rust, not a checked exception, because it's not something you'd want to handle everywhere it can happen, but rather something to propagate up the stack transparently.

Re: Concurrency in Rust

#88

Post by psi-squared » Sun, Mar 27, 2016, 7:05 PM UTC

quotemstr wrote: post #79

steveklabnik wrote: post #77

Earlier quoted context omitted.

> No custom allocator can make the task gracefully report failure > instead of panicing. So, first of all, "custom allocators" means two things: * overloading the allocator that's used by liballoc, and the crates that depend on it, like libstd * other allocators entirely The first is described here: https://doc.rust-lang.org/book/custom-allocators.html And the second is still in RFCs: https://github.com/rust-lang/rfc…

> why is panic-ing bad, but an exception good? Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts. Even if abort-on-panic were to be killed as a legal mode of operation, and even if the stigma were to be removed from std::panic::recover, we'd still be left with a language with two error handling st…

There is an exception-like mechanism in Rust, in the form of the "try!" macro. It's a lot more flexible, but somewhat more verbose (Haskell has the same mechanism in a way that looks a lot more like exceptions, so that's not an inherent flaw). The best explanation I've seen is this:

http://www.jonathanturner.org/2015/11/learning-to-try-things...

tl;dr: "Result"s are like exceptions which are caught by default. You can (explicitly) propagate them upwards by using try!(...). This is nice because it means that you can tell what exceptions can occur in a block of code only using "local" information.

Re: Concurrency in Rust

#89

Post by quotemstr » Sun, Mar 27, 2016, 7:06 PM UTC

Manishearth wrote: post #85

quotemstr wrote: post #79

Earlier quoted context omitted.

> why is panic-ing bad, but an exception good? Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts. Even if abort-on-panic were to be killed as a legal mode of operation, and even if the stigma were to be removed from std::panic::recover, we'd still be left with a language with two error handling st…

> Because the Rust people don't believe in making "catch" a first-class primitive in the language, and in fact, fully support a runtime option to turn all panics into aborts. recover() exists. You're right, there's a stigma to it, because you're not supposed to use it unless you really need to (hence, no programmer confusion). It's supposed to be used for situations like: - Catching panics before crossing an FFI boun…

The problem with the dualistic error handling strategy you're proposing is that the "severe" path gets even less testing than normal error recovery schemes do. Imagine you're working with a big non-exceptional C++ codebase (e.g., Firefox) and somebody throws std::bad_alloc. Even if you don't abort immediately and let the exception unwind the stack, the unwinding process will still leave lots of invariants broken, since all the cleanup paths are wired to return codes and will not run on unwinding.

The result is that your program can be almost arbitrarily broken after throwing. You might as well have just called longjmp.

It's because unwinding in only rare cases often produces bad results that I favor making unwinding the only error-reporting machinery in a language. If you use exceptions to report all errors, everyone starts caring about exception safety again.

Re: Concurrency in Rust

#90

Post by Rusky » Sun, Mar 27, 2016, 7:08 PM UTC

quotemstr wrote: post #86

Manishearth wrote: post #84

Earlier quoted context omitted.

> The world is not Linux. Sure, but if linux has this issue, then C++ programs on linux will also have this issue, and the language can't solve that. That's all my point was. > or to change the signature of all memory-allocation function to return Result. When custom allocators part 2 happens, you can. I've already argued the "real exceptions" part above. > Rust is supposed to be a general-purpose systems programming…

> Sure, but if linux has this issue, then C++ programs on linux will also have this issue Why even bring Linux into the discussion? A Rust program running on Windows has the same problem. > No. My point was simply that no language has a complete solution to this problem. A correct C++ program running on Windows will not spuriously abort. Neither will a C++ program running on a Linux system configured not to overcommi…

...and the same is true of Rust, just with different defaults. Switch allocation failure from an abort to a panic and catch the panic just like you would in C++.

What's incomplete about that? It's not like C++ can't be switched in the other direction with -fno-exceptions or equivalent.