Rust: Dropping heavy things in another thread can make your code 10000x faster • Hacker News Board

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#101

Post by estebank » Sat, May 30, 2020, 6:57 PM UTC

papaf wrote: post #79

fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

In Rust you could just call `mem::forget` on whatever heavy thing that you're no longer using is before it would get dropped, but then the programmer is effectively responsible for that memory leak not becoming a problematic leak during refactors.

Edit: this will also break any code that relies on Drop being called for clean up, but that is already a "suspect"/incorrect pattern because there are no assurances that it will ever run.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#102

Post by devit » Sat, May 30, 2020, 7:05 PM UTC

andrewfromx wrote: post #3

hmm my first thought its, having to do that is a lot like c and cleaning up my own allocations. This feels like something rust should automatically do for me?

Because it's impossible to do this automatically in the general case.

In particular, types may not be sendable to other threads, or may have side effects on dropping, and in those cases you would need to rearchitect the code before you can apply this technique.

Also this technique adds overhead, so it should never be used (including not doing it conditionally) if you don't care about latency or if the objects are always small, and the compiler cannot know whether that is the case.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#103

Post by estebank » Sat, May 30, 2020, 7:07 PM UTC

pmontra wrote: post #93

firethief wrote: post #51

Earlier quoted context omitted.

Why can't it cleanup right after the work?

Or no cleanup at all. A CLI command that runs for a very short time can allocate memory to perform its job, print the result and exit. Then the OS releases all the memory of the process. No idea if Rust can work like this.

std::mem::forget, which doesn't run destructors:

https://doc.rust-lang.org/std/mem/fn.forget.html

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#104

Post by SilasX » Sat, May 30, 2020, 7:07 PM UTC

Completely different dynamic (because no Rust GC), but this reminds me of how Twitch made their server, written in Go, a lot faster by allocating a bunch of dummy memory at the beginning so the garbage collector doesn't trigger nearly as often:

https://news.ycombinator.com/item?id=21670110

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#105

Post by estebank » Sat, May 30, 2020, 7:11 PM UTC

seventh-chord wrote: post #38

epage wrote: post #20

Earlier quoted context omitted.

I believe this is contrived to prove a point. And this isn't just a help in these contrived examples. I believe process cleanup (an extreme case of cleaning up objects) is one of cases where garbage collection performs better because it doesn't have to unwind the stack, call cleanup functions that are not in the cache, and make a lot of `free` calls to the allocator. I vaguely remember reading about Google killing pr…

Killing a process without freeing all allocations is, as far as I can tell, routine in C. Especially for memory it makes no sense "freeing" allocations, the whole memory space is getting scrapped anyways. Of course, once you add RAAI the compiler cant reason about which destructors it can skip on program exit, and if programmers are negligent of this you get programs that are slow to close.

> Killing a process without freeing all allocations is, as far as I can tell, routine in C.

Many times by accident :)

> if programmers are negligent of this you get programs that are slow to close.

I wouldn't call that negligence, just not fully optimized.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#106

Post by cperciva » Sat, May 30, 2020, 7:12 PM UTC

bszupnick wrote: post #33

cperciva wrote: post #9

If freeing the data structure in question takes this long, how much time are you wasting duplicating the data structure?

This code doesn't duplicate it. In Rust when a variable is sent as an argument to a function it's "ownership" moves to be in the scope of that function. https://doc.rust-lang.org/book/ch04-01-what-is-ownership.htm...

You're missing my point. Unless the only thing you want to do with your giant data structure is measure its size, you're not going to be passing ownership of your only copy of it into the get_size function. You're going to be passing in a copy -- hence the cost of duplicating everything.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#107

Post by pron » Sat, May 30, 2020, 7:13 PM UTC

Jasper_ wrote: post #41

chowells wrote: post #31

This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. Generational/compacting GC has the opposite problem. Garbage collection takes time proportional to the live set, and the amount…

One of my favorite papers by Bacon et al expands on this intuition that garbage collection and reference counting are opposite tradeoffs in many respects, and gives a formal theory for it. My views on gc/rc haven't been the same since. http://researcher.watson.ibm.com/researcher/files/us-bacon/B...

That's a great paper, but one important thing to point out is that some production-grade tracing GCs are on the sophisticated end of that paper, while almost all reference counting GCs are on the rather primitive end. Given the same amount of effort, it's easier to get a reasonable result with reference-counting, but there are industrial-strength tracing GCs out there that have had a lot of effort put into them.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#108

Post by arendtio » Sat, May 30, 2020, 7:14 PM UTC

andreygrehov wrote: post #71

Does anyone know how would this work in Go?

I have no idea, but my guess is that it doesn't matter, as the deallocation is being done by the garbage collection.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#109

Post by projektfu » Sat, May 30, 2020, 7:15 PM UTC

papaf wrote: post #79

fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

Yeah, I like Apple’s (Next’s) approach of pool allocation for each run through the event loop. Defer dealloc, drop pool at the end.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#110

Post by pron » Sat, May 30, 2020, 7:15 PM UTC

zucker42 wrote: post #94

arcticbull wrote: post #65

Earlier quoted context omitted.

How are finalizers invoked if the structure isn't traversed? Would it just be optimized away none of the objects have finalizers? Hence my suggestion about the area allocators being a better point of comparison.

Java is an example of a language with a generational copy collector by default. Most objects in Java don't have a finalizer, since after all the main point of, for example, destructors in C++ is to make sure you don't leak memory, which the GC solves. But when the `finalize` method is used is causes significant overhead. > Objects with finalizers (those that have a non-trivial finalize() method) have significant over…

Which is why finalizers in Java have been officially deprecated [1] and might be removed altogether in a future release.

[1]: https://docs.oracle.com/en/java/javase/14/docs/api/java.base...