Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Concurrency in Rust

doc.rust-lang.org

Post reply on HN
Print view View original

111–120 of 160 posts

Re: Concurrency in Rust

#111

Post by quotemstr »

pcwalton wrote: post #109
quotemstr wrote: post #75

Earlier quoted context omitted.

No custom allocator can make a task that fails to allocate gracefully report an error. Rust's error handling design is just terrible, and mostly a consequence of eschewing exceptions. Had Rust opted for exceptions, it'd be a much better, and actually usable, language. Rust's terribly error-handling strategy is the chief reason not to use it.

> No custom allocator can make a task that fails to allocate gracefully report an error. Yes, you can. You can panic the thread, and you can recover from panics. But honestly this is never enough to gracefully recover from OOM. I can't think of any software that uses exceptions to gracefully recover from OOM (i.e. without crashing the process) in a way that works. How many Java applications do you see that catch OutO…

> How many Java applications do you see that catch OutOfMemoryError on a fine-grained level?

I wrote exactly that sort of code, in Java, last week.

Re: Concurrency in Rust

#112

Post by pcwalton »

quotemstr wrote: post #100
Rusky wrote: post #97

Earlier quoted context omitted.

Even C++ doesn't have mandatory exception support, and even Rust can catch panics from failure-oblivious code.

> Even C++ doesn't have mandatory exception support Yes it does. That some compilers provide a way to disable mandatory language features is no argument. > even Rust can catch panics from failure-oblivious code. Not while maintaining that code's invariants it can't.

> Yes it does. That some compilers provide a way to disable mandatory language features is no argument.

It's actually very relevant that huge amounts of C++ deployed in the world use -fno-exceptions, and many shops (for example, Google!) have a policy of "we do not use exceptions". I don't care about how well languages handle OOM in theory; what matters is how well they handle it in practice.

Re: Concurrency in Rust

#113

Post by pcwalton »

quotemstr wrote: post #111
pcwalton wrote: post #109

Earlier quoted context omitted.

> No custom allocator can make a task that fails to allocate gracefully report an error. Yes, you can. You can panic the thread, and you can recover from panics. But honestly this is never enough to gracefully recover from OOM. I can't think of any software that uses exceptions to gracefully recover from OOM (i.e. without crashing the process) in a way that works. How many Java applications do you see that catch OutO…

> How many Java applications do you see that catch OutOfMemoryError on a fine-grained level? I wrote exactly that sort of code, in Java, last week.

I believe you that you do. That doesn't change the fact that few people actually do it or need to do it. There are many more people who think they need to recover from OOM but actually don't and would be better off if they didn't try. (There have been many security vulnerabilities that have resulted from attempting to handle OOM gracefully that wouldn't have been an issue if malloc just aborted the process.)

Re: Concurrency in Rust

#114

Post by quotemstr »

pcwalton wrote: post #113
quotemstr wrote: post #111

Earlier quoted context omitted.

> How many Java applications do you see that catch OutOfMemoryError on a fine-grained level? I wrote exactly that sort of code, in Java, last week.

I believe you that you do. That doesn't change the fact that few people actually do it or need to do it. There are many more people who think they need to recover from OOM but actually don't and would be better off if they didn't try. (There have been many security vulnerabilities that have resulted from attempting to handle OOM gracefully that wouldn't have been an issue if malloc just aborted the process.)

I agree with you, but a general-purpose systems programming language needs to let _me_ make that determination. It can't abort on my behalf for my own good. It's depressing that Java, of all things, does a better job in this respect than Rust.

And we probably shouldn't be writing so much software in general-purpose systems programming languages.

Re: Concurrency in Rust

#115

Post by pcwalton »

quotemstr wrote: post #114
pcwalton wrote: post #113

Earlier quoted context omitted.

I believe you that you do. That doesn't change the fact that few people actually do it or need to do it. There are many more people who think they need to recover from OOM but actually don't and would be better off if they didn't try. (There have been many security vulnerabilities that have resulted from attempting to handle OOM gracefully that wouldn't have been an issue if malloc just aborted the process.)

I agree with you, but a general-purpose systems programming language needs to let _me_ make that determination. It can't abort on my behalf for my own good. It's depressing that Java, of all things, does a better job in this respect than Rust. And we probably shouldn't be writing so much software in general-purpose systems programming languages.

> I agree with you, but a general-purpose systems programming language needs to let _me_ make that determination. It can't abort on my behalf for my own good.

You can decide. You can use the standard library and deal with exceptions, or you can not use the standard library and deal with malloc failure yourself. The Rust standard library is opinionated in this regard, because it's rarely ever a good idea to try to recover from malloc failure for userland processes.

That said, with recover, which will probably be stabilized, you can recover from malloc problems, which are turned into panics. But I'm sure you know that this can be unreliable on Linux with the default overcommit turned on, and so forth. https://doc.rust-lang.org/std/panic/fn.recover.html

Note all of the debate on the linked issue as to whether recover is a good idea. Most of the Rust community is very hesitant to even allow catching panics at all; they certainly don't find the current situation "unusable".

> It's depressing that Java, of all things, does a better job in this respect than Rust.

I think that Java shouldn't throw exceptions on OOM. It should just abort the process.

Re: Concurrency in Rust

#116

Post by quotemstr »

pcwalton wrote: post #115
quotemstr wrote: post #114

Earlier quoted context omitted.

I agree with you, but a general-purpose systems programming language needs to let _me_ make that determination. It can't abort on my behalf for my own good. It's depressing that Java, of all things, does a better job in this respect than Rust. And we probably shouldn't be writing so much software in general-purpose systems programming languages.

> I agree with you, but a general-purpose systems programming language needs to let _me_ make that determination. It can't abort on my behalf for my own good. You can decide. You can use the standard library and deal with exceptions, or you can not use the standard library and deal with malloc failure yourself. The Rust standard library is opinionated in this regard, because it's rarely ever a good idea to try to rec…

I profoundly disagree with your assertions about the correct way to handle malloc failure. While abort may be acceptable for some specific applications, general-purpose systems don't get to impose that opinion on programmers. Memory is a just another resource, and programs need to deal with resource exhaustion generally. Do you think programs should abort when the disk fills up?

Re: Concurrency in Rust

#117

Post by quotemstr »

pcwalton wrote: post #112
quotemstr wrote: post #100

Earlier quoted context omitted.

> Even C++ doesn't have mandatory exception support Yes it does. That some compilers provide a way to disable mandatory language features is no argument. > even Rust can catch panics from failure-oblivious code. Not while maintaining that code's invariants it can't.

> Yes it does. That some compilers provide a way to disable mandatory language features is no argument. It's actually very relevant that huge amounts of C++ deployed in the world use -fno-exceptions, and many shops (for example, Google!) have a policy of "we do not use exceptions". I don't care about how well languages handle OOM in theory; what matters is how well they handle it in practice.

> for example, Google!

Google's C++ coding standards have done tremendous harm to the C++ community by perpetuating obsolete programming practices like two-phase initialization and lossy error reporting. Google's C++ standards also teach people that it's okay to use the STL and not worry about allocation failure, which hurts program robustness generally.

I'm not the only one who thinks so: see https://www.linkedin.com/pulse/20140503193653-3046051-why-go...

My C++ code is exceptional, modern, and robust, and anyone using -fno-exceptions can go fly a kite.

Re: Concurrency in Rust

#118

Post by tetromino_ »

pcwalton wrote: post #113
quotemstr wrote: post #111

Earlier quoted context omitted.

> How many Java applications do you see that catch OutOfMemoryError on a fine-grained level? I wrote exactly that sort of code, in Java, last week.

I believe you that you do. That doesn't change the fact that few people actually do it or need to do it. There are many more people who think they need to recover from OOM but actually don't and would be better off if they didn't try. (There have been many security vulnerabilities that have resulted from attempting to handle OOM gracefully that wouldn't have been an issue if malloc just aborted the process.)

> That doesn't change the fact that few people actually do it or need to do it.

I am one of those few people. We are running some scientific code (currently, unfortunately, written in C++) on a heterogeneous bunch of compute nodes. Some computations can be extremely memory-intensive, and sometimes in ways that we didn't predict. So it's useful to be able to fail gracefully and record that computation X on node Y with input parameters [Z] failed specifically due to running out of memory at step W - so that e.g. the queue manager can try relaunching the computation on a beefier node or adjusting how many instances of which computation are allowed on Y.

Re: Concurrency in Rust

#119

Post by pcwalton »

quotemstr wrote: post #116
pcwalton wrote: post #115

Earlier quoted context omitted.

> I agree with you, but a general-purpose systems programming language needs to let _me_ make that determination. It can't abort on my behalf for my own good. You can decide. You can use the standard library and deal with exceptions, or you can not use the standard library and deal with malloc failure yourself. The Rust standard library is opinionated in this regard, because it's rarely ever a good idea to try to rec…

I profoundly disagree with your assertions about the correct way to handle malloc failure. While abort may be acceptable for some specific applications, general-purpose systems don't get to impose that opinion on programmers. Memory is a just another resource, and programs need to deal with resource exhaustion generally. Do you think programs should abort when the disk fills up?

Most programs do not need to deal with memory exhaustion. There's often little that can be done other than terminating anyway, many OS configurations remove your ability to effectively recover from it (overcommit and swapping making your app unusably slow so you would better off terminating), and adding rarely tested code paths is a good way to introduce bugs and vulnerabilities.

Programs should abort when the disk fills up due to swap exhaustion, yes. They shouldn't abort if I/O fails, but that's because (a) I/O failure potential occurs in many fewer places than memory allocation failure potential, so it's easier to test; (b) I/O failure can occur for many reasons other than disk space exhaustion, and it's usually fine to handle disk space exhaustion the same way you handle other types of I/O failure, so it isn't any extra burden to handle that case.

Re: Concurrency in Rust

#120

Post by pcwalton »

tetromino_ wrote: post #118
pcwalton wrote: post #113

Earlier quoted context omitted.

I believe you that you do. That doesn't change the fact that few people actually do it or need to do it. There are many more people who think they need to recover from OOM but actually don't and would be better off if they didn't try. (There have been many security vulnerabilities that have resulted from attempting to handle OOM gracefully that wouldn't have been an issue if malloc just aborted the process.)

> That doesn't change the fact that few people actually do it or need to do it. I am one of those few people. We are running some scientific code (currently, unfortunately, written in C++) on a heterogeneous bunch of compute nodes. Some computations can be extremely memory-intensive, and sometimes in ways that we didn't predict. So it's useful to be able to fail gracefully and record that computation X on node Y with…

That's something that Rust fully supports with panic handlers. Doing arbitrary work before the process goes down is useful and supported. (But you will have to be running Linux in a non-default configuration for it to be reliable, of course!)
Post reply on HN