Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

61–70 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#61

Post by fpgaminer »

Some important things I think people should note before blindly commenting:

* The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world.

* I didn't see any difference on my machine between a debug build and a release build.

* The example is preforming 1 _million_ deallocations. That's why it's so pathological. It's not just a "large" vector. It's a vector of 1 million vectors. While that may seem contrived, consider a vector of 1 million strings, something that's not too uncommon, and which would likely suffer the same performance penalty.

* Rust is not copying anything, nor duplicating the structures here. In the example code the structures would be moved, not copied, which costs nothing. The deallocation is taking up 99% of the time.

* As an aside, compilers have used the trick of not free-ing data structures before, because it provides a significant performance boost. Instead of calling free on all those billions of tiny data structures a compiler would generate during its lifetime, they just let them leak. Since a compiler is short lived its not a problem, they get a free lunch (pun unintended), and the OS takes care of cleaning up after all is said and done. My point is that this post isn't theoretical, we do deallocation trickery in the real world.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#62

Post by chowells »

arcticbull wrote: post #40
chowells wrote: post #31

This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. Generational/compacting GC has the opposite problem. Garbage collection takes time proportional to the live set, and the amount…

> This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. That doesn't seem to make intuitive sense. A GC has the same problem. A garbage collector has to traverse the data structure…

I said generational/compacting collector. You're talking about a mark and sweep collector.

A generational/compacting collector traverses pointers from the live roots, and copies everything it finds to the start of its memory space, and then declares the rest unused. If there is 1GB of unused memory, it's irrelevant. Only the things that can be reached are even examined.

As I said, this has the opposite problem. When the live set becomes huge, this can drag performance. When the live set is small, it doesn't matter how much garbage it produces, performance is fast.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#63

Post by pornel »

Ididntdothis wrote: post #34
saagarjha wrote: post #27

Earlier quoted context omitted.

Rust basically gives the compiler understanding of unique_ptr and prevents you from using it after you’ve moved it.

Would you have to keep track of these threads in Rust? I have done a lot of desktop development where you have to be aware of what happens during shutdown. Seems a lot of server guys write their code under the assumption that it will never shut down.

You would need to add `thread.join()` at the end of main, or have some RAII guard that does it for you.

In practice that's probably optional, because the heap and all resources are usually torn down with the process anyway. Important things, like saving data or committing transactions, shouldn't be done in destructors.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#64

Post by arcticbull »

Reelin wrote: post #54
arcticbull wrote: post #40

Earlier quoted context omitted.

> This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. That doesn't seem to make intuitive sense. A GC has the same problem. A garbage collector has to traverse the data structure…

> The only way to drop an extensively nested structure like this any faster than traversing it would be an arena allocator, and forgetting about the entire arena. Isn't that incompatible with RAII though?

You can handle this in Rust pretty neatly with lifetimes. There's a bunch of crates that do this. [1]

[1] https://crates.io/crates/typed-arena

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#65

Post by arcticbull »

chowells wrote: post #62
arcticbull wrote: post #40

Earlier quoted context omitted.

> This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. That doesn't seem to make intuitive sense. A GC has the same problem. A garbage collector has to traverse the data structure…

I said generational/compacting collector. You're talking about a mark and sweep collector. A generational/compacting collector traverses pointers from the live roots, and copies everything it finds to the start of its memory space, and then declares the rest unused. If there is 1GB of unused memory, it's irrelevant. Only the things that can be reached are even examined. As I said, this has the opposite problem. When…

How are finalizers invoked if the structure isn't traversed? Would it just be optimized away none of the objects have finalizers? Hence my suggestion about the area allocators being a better point of comparison.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#66

Post by cesarb »

Just be careful, because moving heavy things to be dropped to another thread can change the semantics of the program. For instance, consider what happens if within that heavy thing you had a BufWriter: unless its buffer is empty, dropping it writes the buffer, so now your file is being written and closed in a random moment in the future, instead of being guaranteed to have been sent to the kernel and closed when the function returns.

And it can even be worse if it's holding a limited resource, like a file descriptor or a database connection. That is, I wouldn't recommend using this trick unless you're sure that the only thing the "heavy thing" is holding is memory (and even then, keep in mind that memory can also be a limited resource).

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#67

Post by DasIch »

cs702 wrote: post #56

In other words, Rust's automagical memory deallocation is NOT a zero-cost abstraction: fn get_len1(things: HeavyThings) -> usize { things.len() } fn get_len2(things: HeavyThings) -> usize { let len = things.len(); thread::spawn(move || drop(things)); len } The OP shows an example in which a function like get_len2 is 10000x faster than a function like get_len1 for a hashmap with 1M keys. See also this comment by chowe…

Nothing about how Rust handles deallocation is magical in any way.

It's also definitely a zero-cost abstraction as I can see because the manual solution that's equivalent to get_len1() would be to essentially call free() on things. That would ultimately suffer from the same problem.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#68

Post by zozbot234 »

Ididntdothis wrote: post #24

I used to do this sometimes with C++ when I realized that clearing out a vector with lots of objects was slow. Is Rust basically based on unique_ptr? One problem with this approach was that you still had to wait for these threads when the application would shut down.

> One problem with this approach was that you still had to wait for these threads when the application would shut down.

If you know that an object will live for the rest of the program and not need any finalization logic, Rust allows you to "leak" it and save that overhead on shutdown.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#69

Post by renewiltord »

sockgrant wrote: post #55
dirtydroog wrote: post #4

Oh my good god. I'm hoping this is down to developer naivety rather than being a feature of rust.

1) he should pass by reference to avoid the extra copy. So in his example yes it’s dev naivety 2) but somewhere somehow this object will deallocate, so his trick of putting it to another thread would work if the deal location takes awhile. Same for cpp if you have a massive object in a unique ptr. So it’s not a rust issue

Where's the extra copy? I don't see one. He's moving the struct into the function, getting size and then dropping it.
Post reply on HN