Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

141–150 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#141

Post by Reelin »

jacobparker wrote: post #95
Reelin wrote: post #39

Earlier quoted context omitted.

> the ownership system lets you transfer freeing responsibility off-thread safely and cheaply in order to not have it block the critical path This can also trivially be done in other languages. Atomically append your pointer to a queue of "large things that need to be freed" and move on as though you had actually called free. Within a particularly time sensitive loop you can even opt to place pointers into a prealloc…

OP said safely; what you're describing isn't safe in, say, C++ in the same sense that it is in Rust.

Isn't that essentially a tautology? Manual memory management and threading in such languages lacks safety guarantees to begin with.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#142

Post by the8472 »

SilasX wrote: post #104

Completely different dynamic (because no Rust GC), but this reminds me of how Twitch made their server, written in Go, a lot faster by allocating a bunch of dummy memory at the beginning so the garbage collector doesn't trigger nearly as often: https://news.ycombinator.com/item?id=21670110

The java equivalent to the Go case would simply be adjusting the -Xms flag. The Go approach is a needlessly convoluted because the runtime doesn't offer any tuning knobs.

As for the rust case, if you squint then it's similar to a concurrent collector.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#143

Post by rubber_duck »

ycombobreaker wrote: post #139
rubber_duck wrote: post #100

Earlier quoted context omitted.

>As an aside, compilers have used the trick of not free-ing data structures before, because it provides a significant performance boost. Instead of calling free on all those billions of tiny data structures a compiler would generate during its lifetime, they just let them leak. Since a compiler is short lived its not a problem, they get a free lunch (pun unintended), and the OS takes care of cleaning up after all is…

In a world where processes can fork-and-exec, nothing about "as a service" changes that. The compiler would just be reinvoked as needed. Converting it into a persistent process breaks a lot more than just allocation optimizations.

But you want to share state and only update state incrementally on edit to get any reasonable level of performance for stuff like language server code analysis.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#144

Post by MaxBarraclough »

mcguire wrote: post #120
projektfu wrote: post #109

Earlier quoted context omitted.

Yeah, I like Apple’s (Next’s) approach of pool allocation for each run through the event loop. Defer dealloc, drop pool at the end.

Unless your destructors do more than deallocation, in which case you will leak whatever other resource you're managing.

A pool can invoke destructors when it is cleared. Might take a bit of overhead (if the pool is to support arbitrary classes), but you could retain the fast pointer-bump allocation.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#145

Post by kevincox »

justinpombrio wrote: post #129
indemnity wrote: post #126

Earlier quoted context omitted.

Isn’t a Rust “move” implemented as a bit wise copy (e.g. memcpy call)? I see people claiming move has no cost but I’m not sure that is true.

EDIT: See kevincox's reply. Rust will bitwise copy the containing type , which is typically very cheap. For example, it you move a String, it will copy the String struct, which contains a couple pointers and a length (or something along those lines). Importantly, it will not copy the underlying char array. I was thinking of the following code, where I believe the assignment to y is actually free. Though apparently th…

Semantically it is a bitwise copy.

In the example from the article it is probably actually a copy because the value was originally on the parent thread's stack, which will be reused after the function returns, so the value will need to be copied to the new thread's stack.

However it is important to not that it isn't a deep/recursive bitwise copy. It just needs to copy the HashMap itself (which is probably a handful of words).

So yes, it is doing a bitwise copy, but this is also very cheap. It will be much, much cheaper than spawning the thread.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#146

Post by the8472 »

cesarb wrote: post #66

Just be careful, because moving heavy things to be dropped to another thread can change the semantics of the program. For instance, consider what happens if within that heavy thing you had a BufWriter: unless its buffer is empty, dropping it writes the buffer, so now your file is being written and closed in a random moment in the future, instead of being guaranteed to have been sent to the kernel and closed when the…

Considering that writing files can also block the process you probably don't want to have that in your latency-sensitive parts either, so you'll have to optimize that one way or another anyway.

For the more general problem you have can also dedicate more threads to the task or apply backpressure.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#147

Post by dathinab »

One thing I just noticed is that the example doesn't make sure to actually run the new thread to completion before the main thread exists.

This means that if you do a "drop in other thread" and then main exists, the drop might never run. Which is often fine as the exit of main causes process termination and as such will free the memory normally anyway.

But it would be a problem one some systems where memory cleanup on process exit is less reliable. Through such systems are more rare by now I think.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#148

Post by rubber_duck »

yrro wrote: post #127
rubber_duck wrote: post #100

Earlier quoted context omitted.

>As an aside, compilers have used the trick of not free-ing data structures before, because it provides a significant performance boost. Instead of calling free on all those billions of tiny data structures a compiler would generate during its lifetime, they just let them leak. Since a compiler is short lived its not a problem, they get a free lunch (pun unintended), and the OS takes care of cleaning up after all is…

As distasteful as leaky code is, is it that bad to run it in a separate process? You get a bit more robustness against crashes as well.

You want to be able to incrementally update state to get performance out of incremental code analysis (eg. language server implemention for IDE)

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#149

Post by littlestymaar »

The title is slightly wrong: it's not going to make your code faster, it's going to reduce latency on the given thread.

It maybe a net win if this is the UI thread of a desktop app, but overall, it will come at a performance cost: because modern allocators have thread-local memory pools, and now you're moving away from it. And if you're running you code on a NUMA system (most server nowadays), when moving from one thread to another, you can end up freeing non-local memory instead of local one. Also, you won't have any backpressure on your allocations, and you are susceptible to run out of memory (especially because your deallocations now occur more slowly than they should)

Main takeaway: if you use it blindly it's an anti-pattern, but it can be a good idea in its niche: the UI thread of a GUI.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#150

Post by ReactiveJelly »

dathinab wrote: post #147

One thing I just noticed is that the example doesn't make sure to actually run the new thread to completion before the main thread exists. This means that if you do a "drop in other thread" and then main exists, the drop might never run. Which is often fine as the exit of main causes process termination and as such will free the memory normally anyway. But it would be a problem one some systems where memory cleanup o…

It would have to be a non-desktop system.

I'm pretty sure Linux will always free process-private memory, and threads, and file descriptors when a process exits.

The only things that can leak in typical cases are some kinds of shared memory and maybe child processes?

Post reply on HN