Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

181–190 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#181

Post by folmar »

pdimitar wrote: post #172
rcxdude wrote: post #170

Earlier quoted context omitted.

Deallocation at the end of a program's execution can substantially add to its runtime, and it's entirely waste. It's a much more common strategy than you might think.

You are right, I indeed didn't know it was that common. But still, in a world where languages and runtimes are also judged by their ability to run in lambda/serverless setups, I'd think this practice will start being obsolete, wouldn't you think? (What I mean is that I imagine that any serverless function that runs in severely constrained and measured environments like the AWS Lambda would gain a significant edge ove…

Most compiler are not designed to run in daemon mode, specifically it's non-issue since their startup is normally fast. And the compiler and runtime are a different thing.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#182

Post by ncmncm »

There is nothing unique to Rust about this; it is a very old technique. It is usually much inferior to the "arena allocator" method, where all the discarded allocations are coalesced and released in a single, cheap operation that could as well be done without another thread. That method is practical in many languages, Rust possibly included. C++ supports it in the Standard Library, for all the standard containers.

If important work must be done in the destructors, it is still better to farm the work out to a thread pool, rather than starting another thread. Again, C++ supports this in its Standard Library, as I think Rust does too.

One could suggest that the only reason to present the idea in Rust is the cynical one that Rust articles get free upvotes on HN.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#183

Post by pdimitar »

thebean11 wrote: post #178
pdimitar wrote: post #166

Earlier quoted context omitted.

I am suggesting they apply good practices. I'd never imagine that compilers were actually doing what was stated -- sounds awful. I understand it's tradeoffs and we all have real-world limitations to contend with -- but again, of all the corners that could be cut that's exactly the one I didn't imagine they would. Nasty.

Can you articulate why it's a bad practice? If it works better than alternatives and it's documented, not really sure what the issue is. I don't think it's even that uncommon. I believe some HFT firms run Java with a huge amount of RAM and GC disabled, and get around it by just rebooting the software occasionally. To me writing software like that is fair game, I don't see the point in being dogmatic about "how things…

Mostly because I look at it from the angle of one-off / general purpose / CLI programs. If one such has to run for 10-30 seconds and its memory just keeps growing and growing with the idea of throwing it all away at the end and letting the OS handle it, it might become disruptive for other programs on the machine.

For specialised apps and servers it's of course a perfectly good practice.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#184

Post by pdimitar »

folmar wrote: post #181
pdimitar wrote: post #172

Earlier quoted context omitted.

You are right, I indeed didn't know it was that common. But still, in a world where languages and runtimes are also judged by their ability to run in lambda/serverless setups, I'd think this practice will start being obsolete, wouldn't you think? (What I mean is that I imagine that any serverless function that runs in severely constrained and measured environments like the AWS Lambda would gain a significant edge ove…

Most compiler are not designed to run in daemon mode, specifically it's non-issue since their startup is normally fast. And the compiler and runtime are a different thing.

I realise that, but nowadays language servers are a pretty normal practice in no small amount of areas.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#185

Post by folmar »

rubber_duck wrote: post #148
yrro wrote: post #127

Earlier quoted context omitted.

As distasteful as leaky code is, is it that bad to run it in a separate process? You get a bit more robustness against crashes as well.

You want to be able to incrementally update state to get performance out of incremental code analysis (eg. language server implemention for IDE)

There are more ways to go then that: AST fragment caching, intermediate representation fragment caching and so on. Incremental updates fit some languages better than the others.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#186

Post by folmar »

wongarsu wrote: post #167
papaf wrote: post #79

Earlier quoted context omitted.

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

In a toy raytracer I once wrote in C++, switching from malloc to custom memory pools for small fixed-size objects was a big performance boost. Making free() a noop was another big performance boost, both for deallocation and allocation. Turns out sequentially handing out memory from a big chunk of memory is much easier than keeping track of and reusing empty slots, and it keeps sequentially allocated objects in seque…

jemalloc is the answer to the question you did not ask

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#187

Post by jashmatthews »

rubber_duck wrote: post #140
qzw wrote: post #132

Earlier quoted context omitted.

Well, then that’s not the original use case anymore, and it’ll have to be re-engineered. In the meantime it may have been used for years and the perf difference may have saved many developer-years collectively across its user base. Surely you’re not suggesting that the compiler developers should be prematurely optimizing for future use cases that they may not even have envisioned.

Avoiding leaks is not optimisation, it's a matter of correctness - not freeing memory is an optimisation based on a very shortsighted assumption that is not practical for any new language (modern languages are expected to come with language server support)

Why do you say that? Even if you call free immediately after a piece of memory is no longer needed, malloc won’t release that immediately anyway.

If this is incorrect, then every modern malloc implementation is incorrect.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#188

Post by hinkley »

rini17 wrote: post #168
hinkley wrote: post #122

Earlier quoted context omitted.

One of my “favorite” snags in perf analysis is that periodicity in allocations can misattribute the cost of allocations to the wrong function. If I allocate just enough memory, but not too much, then pauses for defragmentation of free space may be costed to the code that calls me. A solution to this that I’ve seen in soft real time systems is to amortize cleanups across all allocations. Every allocation performs n st…

Doing unrelated cleanup sounds like flushing CPU cache per every allocation.

This post is already about unrelated cleanup, so I'm not sure what other escape hatch you imagine people taking. Do you have a suggestion?

You can tell that it's unrelated cleanup because if it were related, then the cost of freeing wouldn't be noteworthy. It would be cache hot because you would be visiting it for the second time. In which case we'd be talking about why you are scanning a giant object on an event loop in the first place. That's not what's at issue. What's at issue is that you've been handed this great bomb of uncached data from someone else and now you're stuck doing the janitorial work.

Freeing an object of arbitrary size is effectively an unbounded operation. Cache invalidation has a very high cost, sure, but it's still bounded.

Putting a limit on the amount of work you do, you could stop before purging the entire cache. You could use a smaller limit on doing work for someone else and control invalidation there, too.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#189

Post by dan-robertson »

bauerd wrote: post #97
orf wrote: post #86

Earlier quoted context omitted.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

Think with LD_PRELOAD it is always possible to override the allocator?

Only if it is dynamically linked

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#190

Post by jpitz »

rubber_duck wrote: post #140
qzw wrote: post #132

Earlier quoted context omitted.

Well, then that’s not the original use case anymore, and it’ll have to be re-engineered. In the meantime it may have been used for years and the perf difference may have saved many developer-years collectively across its user base. Surely you’re not suggesting that the compiler developers should be prematurely optimizing for future use cases that they may not even have envisioned.

Avoiding leaks is not optimisation, it's a matter of correctness - not freeing memory is an optimisation based on a very shortsighted assumption that is not practical for any new language (modern languages are expected to come with language server support)

Correctness means adherence to the spec, not some contrived absolute truth.
Post reply on HN