Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

91–100 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#91

Post by cs702 »

DasIch wrote: post #67
cs702 wrote: post #56

In other words, Rust's automagical memory deallocation is NOT a zero-cost abstraction: fn get_len1(things: HeavyThings) -> usize { things.len() } fn get_len2(things: HeavyThings) -> usize { let len = things.len(); thread::spawn(move || drop(things)); len } The OP shows an example in which a function like get_len2 is 10000x faster than a function like get_len1 for a hashmap with 1M keys. See also this comment by chowe…

Nothing about how Rust handles deallocation is magical in any way. It's also definitely a zero-cost abstraction as I can see because the manual solution that's equivalent to get_len1() would be to essentially call free() on things. That would ultimately suffer from the same problem.

Yeah, you're right. In hindsight this was a poorly thought-out and poorly written post on my part.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#92

Post by cs702 »

dathinab wrote: post #76
cs702 wrote: post #56

In other words, Rust's automagical memory deallocation is NOT a zero-cost abstraction: fn get_len1(things: HeavyThings) -> usize { things.len() } fn get_len2(things: HeavyThings) -> usize { let len = things.len(); thread::spawn(move || drop(things)); len } The OP shows an example in which a function like get_len2 is 10000x faster than a function like get_len1 for a hashmap with 1M keys. See also this comment by chowe…

No the zero-cost refers to the abstraction (and runtime cost), which still is zero-cost. Deallocating is part of the normal work load not the abstraction. Also this isn't rust specific. Most (all?) RAII languages are affected and many GC approaches have this effect, too. Some do add additional abstraction to magically always or sometimes put the de-allocation into another thread. But de-allocating in another thread i…

> No the zero-cost refers to the abstraction (and runtime cost), which still is zero-cost. Deallocating is part of the normal work load not the abstraction.

Yeah, you're right. In hindsight my comment was poorly thought-out and poorly written.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#93

Post by pmontra »

firethief wrote: post #51
epage wrote: post #26

For those wanting a real world example where this can be useful: I am writing a static site generator. When run in "watch" mode, it deletes everything and starts over (I'd like to reduce these with partial updates but can't always do it). Moving that cleanup to a thread would make "watch" more responsive.

Why can't it cleanup right after the work?

Or no cleanup at all. A CLI command that runs for a very short time can allocate memory to perform its job, print the result and exit. Then the OS releases all the memory of the process. No idea if Rust can work like this.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#94

Post by zucker42 »

arcticbull wrote: post #65
chowells wrote: post #62

Earlier quoted context omitted.

I said generational/compacting collector. You're talking about a mark and sweep collector. A generational/compacting collector traverses pointers from the live roots, and copies everything it finds to the start of its memory space, and then declares the rest unused. If there is 1GB of unused memory, it's irrelevant. Only the things that can be reached are even examined. As I said, this has the opposite problem. When…

How are finalizers invoked if the structure isn't traversed? Would it just be optimized away none of the objects have finalizers? Hence my suggestion about the area allocators being a better point of comparison.

Java is an example of a language with a generational copy collector by default. Most objects in Java don't have a finalizer, since after all the main point of, for example, destructors in C++ is to make sure you don't leak memory, which the GC solves. But when the `finalize` method is used is causes significant overhead.

> Objects with finalizers (those that have a non-trivial finalize() method) have significant overhead compared to objects without finalizers, and should be used sparingly. Finalizeable objects are both slower to allocate and slower to collect. At allocation time, the JVM must register any finalizeable objects with the garbage collector, and (at least in the HotSpot JVM implementation) finalizeable objects must follow a slower allocation path than most other objects. Similarly, finalizeable objects are slower to collect, too. It takes at least two garbage collection cycles (in the best case) before a finalizeable object can be reclaimed, and the garbage collector has to do extra work to invoke the finalizer. [1]

Sure, you're technically correct that if the objects all had finalizers that did the same thing as C++ destructors, it would be equivalent, but because of the existence of a GC we don't have to do any work for most objects. A GC is equivalent to an arena allocator in this sense.

Another point is the C++/Rust pattern of each object recursively freeing the objects it owns presumably leads to slower deallocation, because in the general case it involves pointer following and non-local access.

[1] https://www.ibm.com/developerworks/java/library/j-jtp01274/i...

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#95

Post by jacobparker »

Reelin wrote: post #39
chowells wrote: post #31

This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. Generational/compacting GC has the opposite problem. Garbage collection takes time proportional to the live set, and the amount…

> the ownership system lets you transfer freeing responsibility off-thread safely and cheaply in order to not have it block the critical path This can also trivially be done in other languages. Atomically append your pointer to a queue of "large things that need to be freed" and move on as though you had actually called free. Within a particularly time sensitive loop you can even opt to place pointers into a prealloc…

OP said safely; what you're describing isn't safe in, say, C++ in the same sense that it is in Rust.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#96

Post by vvanders »

papaf wrote: post #79
fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

You can also use the typed-arena crate[0] or roll your own if you're feeling like cracking open unsafe.

[0] https://crates.io/crates/typed-arena

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#97

Post by bauerd »

orf wrote: post #86
papaf wrote: post #79

Earlier quoted context omitted.

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

Think with LD_PRELOAD it is always possible to override the allocator?

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#98

Post by CoolGuySteve »

papaf wrote: post #79
fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

Even just keeping a free list and deallocating it’s elements at an idle time is probably cheaper and faster than spawning a thread.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#99

Post by heftig »

If I seriously wanted to move object destruction off-thread, I would use at least a dedicated thread with a channel, so I could make sure the dropper is done at some point (before the program terminates, at the latest). It also avoids starting and stopping threads constantly.

Something like this: https://play.rust-lang.org/?version=stable&mode=debug&editio...

You could have an even more advanced version spawning tasks into something like rayon's thread pool, I assume.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#100

Post by rubber_duck »

fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

>As an aside, compilers have used the trick of not free-ing data structures before, because it provides a significant performance boost. Instead of calling free on all those billions of tiny data structures a compiler would generate during its lifetime, they just let them leak. Since a compiler is short lived its not a problem, they get a free lunch (pun unintended), and the OS takes care of cleaning up after all is said and done. My point is that this post isn't theoretical, we do deallocation trickery in the real world.

And then someone tries to use your compiler as a service (code analysis, change triggered compiler) and it's a dead end

Post reply on HN