Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

221–230 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#221

Post by chubot »

Looks like Evan Wallace ran into the same issue in practice in esbuild

https://news.ycombinator.com/item?id=22336284

I actually originally wrote esbuild in Rust and Go, and Go was the clear winner.

The parser written in Go was both faster to compile and faster to execute than the parser in Rust. The Go version compiled something like 100x faster than Rust and ran at something around 10% faster (I forget the exact numbers, sorry). Based on a profile, it looked like the Go version was faster because GC happened on another thread while Rust had to run destructors on the same thread.

ESBuild is a really impressive performance-oriented project:

https://github.com/evanw/esbuild

The Rust version also had other problems. Many places in my code had switch statements that branched over all AST nodes and in Rust that compiles to code which uses stack space proportional to the total stack space used by all branches instead of just the maximum stack space used by any one branch: https://github.com/rust-lang/rust/issues/34283

(copy of lobste.rs comment)

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#222

Post by cesarb »

pron wrote: post #110
zucker42 wrote: post #94

Earlier quoted context omitted.

Java is an example of a language with a generational copy collector by default. Most objects in Java don't have a finalizer, since after all the main point of, for example, destructors in C++ is to make sure you don't leak memory, which the GC solves. But when the `finalize` method is used is causes significant overhead. > Objects with finalizers (those that have a non-trivial finalize() method) have significant over…

Which is why finalizers in Java have been officially deprecated [1] and might be removed altogether in a future release. [1]: https://docs.oracle.com/en/java/javase/14/docs/api/java.base...

Unfortunately, the alternative they recommend (Cleaner) only exists since Java 9, so it cannot be used by libraries that wish to remain compatible with Java 8 (which, in my experience, is most or all of them; so far, I haven't seen any Java library which has dropped compatibility with Java 8, and many only recently dropped compatibility with Java 6 or 7).

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#223

Post by cesarb »

pdpi wrote: post #136
msclrhd wrote: post #119

Earlier quoted context omitted.

Destructors in C++ aren't just for making sure you leak memory. They are used for many lifetime controlled things such as: 1. general resource cleanup (file handle, database connection, etc.) using RAII (Resource Aquisition Is Initialization); 2. tracing function entry/exit.

In the JVM the equivalent to RAII is implemented with try-with-resource/Autocloseable instead.

In my experience, try-with-resources is inferior to RAII, since it's very easy to forget (that is, it's very easy to do "Foo x = bar();" instead of "try (Foo x = bar()) { ... }"), leading to resource leaks. More than once I have added a finalizer to an AutoCloseable class just to warn loudly if close() hasn't been called; unfortunately, there's no way that I know of to suppress the finalizer once close() is called, so the GC still has to do all the work to call the finalizer even when the class was used correctly.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#224

Post by kevingadd »

littlestymaar wrote: post #149

The title is slightly wrong: it's not going to make your code faster , it's going to reduce latency on the given thread. It maybe a net win if this is the UI thread of a desktop app, but overall, it will come at a performance cost: because modern allocators have thread-local memory pools, and now you're moving away from it. And if you're running you code on a NUMA system (most server nowadays), when moving from one t…

this could make your code faster by providing more consistent control flow: the main thread is always doing Work and your gc threads are always cleaning up dead objects. this provides fewer, better-predicted branches and code that's more likely to stay in the icache.

most gc based environments use dedicated threads for gc and finalizers, this is one reason to do so

edit: to be more specific:

your normal flow is to alloc at the top of your function, and at the bottom you dealloc. so in basically every case you are paying the cost of deallocs, but if the alloc is conditional the dealloc is now also conditional which is more branches to predict. the dealloc is probably also handled by functions so you have jumps/calls eating up branch prediction table space

in the gc/offloaded dealloc scenario, your deallocs on the work thread are no longer conditional because you're just handing addresses off to the gc. if your gc is STW you've added 'if (stop_requested) stop()' branches throughout your workload, but those are effectively 0-cost because stop_requested is always false (when it's true, the cost of the mispredict has no significance because your thread is about to suspend). the gc thread is always doing the same thing or waiting, and again when it's about to wait a branch mispredict cost has no significance.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#225

Post by nelhage »

rubber_duck wrote: post #140
qzw wrote: post #132

Earlier quoted context omitted.

Well, then that’s not the original use case anymore, and it’ll have to be re-engineered. In the meantime it may have been used for years and the perf difference may have saved many developer-years collectively across its user base. Surely you’re not suggesting that the compiler developers should be prematurely optimizing for future use cases that they may not even have envisioned.

Avoiding leaks is not optimisation, it's a matter of correctness - not freeing memory is an optimisation based on a very shortsighted assumption that is not practical for any new language (modern languages are expected to come with language server support)

We used precisely this optimization in [sorbet](https://sorbet.org), a brand-new type checker for Ruby, which also contains a high-performance LSP server.

We wrote the entire thing (and tested, using ASAN and fuzzers and other techniques) to avoid leaking memory, and then strategically inserted [the equivalent of a rust `mem::forget`](https://github.com/sorbet/sorbet/blob/0aae56e73c7680ec6053b3...) into the end of the `main` driver during standalone mode, to avoid calling those destructors when we're about to exit anyways.

This optimization is definitely still relevant for new systems today.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#226

Post by ashtonkem »

elcomet wrote: post #90
epage wrote: post #26

For those wanting a real world example where this can be useful: I am writing a static site generator. When run in "watch" mode, it deletes everything and starts over (I'd like to reduce these with partial updates but can't always do it). Moving that cleanup to a thread would make "watch" more responsive.

That's not really the same issue that is mentionned in the article though, is it ? The issue from the article would be solved by just passing a reference to the variable. In your case, cleanup is an action that needs to be done before writing new files. So you have to wait for cleanup anyway, don't you ?

That's not true.

Typically any server with a watch functionality will have a mutable reference to the data that's being watched. When you change that data out you're both changing the mutable reference, and also deallocating any memory that was previously used. One could separate these two steps, moving the watched data to another variable that's dropped in another thread, if you wanted.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#227

Post by jfkebwjsbx »

bauerd wrote: post #97
orf wrote: post #86

Earlier quoted context omitted.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

Think with LD_PRELOAD it is always possible to override the allocator?

That wouldn’t help. The point is to use local allocators which have extra information about the data structures and usage of the memory.

This is routinely done in medium to large C++ programs for different reasons (performance, debuggability...).

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#228

Post by jfkebwjsbx »

thebean11 wrote: post #178
pdimitar wrote: post #166

Earlier quoted context omitted.

I am suggesting they apply good practices. I'd never imagine that compilers were actually doing what was stated -- sounds awful. I understand it's tradeoffs and we all have real-world limitations to contend with -- but again, of all the corners that could be cut that's exactly the one I didn't imagine they would. Nasty.

Can you articulate why it's a bad practice? If it works better than alternatives and it's documented, not really sure what the issue is. I don't think it's even that uncommon. I believe some HFT firms run Java with a huge amount of RAM and GC disabled, and get around it by just rebooting the software occasionally. To me writing software like that is fair game, I don't see the point in being dogmatic about "how things…

It is bad practice if your code cannot turn on deallocation for debugging purposes or library-like usage.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#229

Post by jfkebwjsbx »

zucker42 wrote: post #94
arcticbull wrote: post #65

Earlier quoted context omitted.

How are finalizers invoked if the structure isn't traversed? Would it just be optimized away none of the objects have finalizers? Hence my suggestion about the area allocators being a better point of comparison.

Java is an example of a language with a generational copy collector by default. Most objects in Java don't have a finalizer, since after all the main point of, for example, destructors in C++ is to make sure you don't leak memory, which the GC solves. But when the `finalize` method is used is causes significant overhead. > Objects with finalizers (those that have a non-trivial finalize() method) have significant over…

> after all the main point of, for example, destructors in C++ is to make sure you don't leak memory, which the GC solves

No, C++ destructors are used for finalizers, not memory management.

Memory deallocation is a particular use case for finalizers which is avoided when performance is a concern.

> Another point is the C++/Rust pattern of each object recursively freeing the objects it owns presumably leads to slower deallocation, because in the general case it involves pointer following and non-local access.

No, a program that does pointer chasing and has to deallocate many small allocations is badly designed. If you are going to do that, using a GC language would be much better.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#230

Post by altfredd »

cesarb wrote: post #222
pron wrote: post #110

Earlier quoted context omitted.

Which is why finalizers in Java have been officially deprecated [1] and might be removed altogether in a future release. [1]: https://docs.oracle.com/en/java/javase/14/docs/api/java.base...

Unfortunately, the alternative they recommend (Cleaner) only exists since Java 9, so it cannot be used by libraries that wish to remain compatible with Java 8 (which, in my experience, is most or all of them; so far, I haven't seen any Java library which has dropped compatibility with Java 8, and many only recently dropped compatibility with Java 6 or 7).

The actual alternative is PhantomReference, which exists since ancient times. Cleaner simply provides safe way to use PhantomReferences, which are rather tricky to use (by Java standards).
Post reply on HN