Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

201–210 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#201

Post by all-fakes »

ckcheng wrote: post #153
fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

>As an aside, compilers have used the trick of not free-ing data structures before, because it provides a significant performance boost. Instead of calling free on all those billions of tiny data structures a compiler would generate during its lifetime, they just let them leak. Since a compiler is short lived its not a problem, they get a free lunch (pun unintended), and the OS takes care of cleaning up after all is…

Yes, Java also has a garbage collector that does nothing, called the Epsilon GC, intended for short-lived programs and references for garbage collector benchmarks.[0]

[0]: https://blogs.oracle.com/javamagazine/epsilon-the-jdks-do-no...

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#202

Post by pdimitar »

setr wrote: post #196
pdimitar wrote: post #172

Earlier quoted context omitted.

You are right, I indeed didn't know it was that common. But still, in a world where languages and runtimes are also judged by their ability to run in lambda/serverless setups, I'd think this practice will start being obsolete, wouldn't you think? (What I mean is that I imagine that any serverless function that runs in severely constrained and measured environments like the AWS Lambda would gain a significant edge ove…

I don't understand; a lambda function executes and closes, and any leak gets handled by lambda runtime (by freeing the whole lambda function)-- exactly the same as running a cli tool. It seems to me that the server-less context is actually exactly where you'd employ this strategy: the programs by definition cannot be long-lived, so unless they generate so much garbage as to oom in that timeline, there is no need to d…

Yeah, this is what I was saying -- that it could produce too much garbage if no manual free-ing of memory is done.

I suppose, judging by the downvotes, that some find that perspective naive so I'll just cut it here because no productive discussion is happening. :(

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#203

Post by all-fakes »

vvanders wrote: post #96
papaf wrote: post #79

Earlier quoted context omitted.

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

You can also use the typed-arena crate[0] or roll your own if you're feeling like cracking open unsafe. [0] https://crates.io/crates/typed-arena

Typed-arena is a bit limited, because all the allocated values must be of the same type. bumpalo[0] removes that restriction, but it also doesn't run destructors, which can lead to memory and other resource leaks. Another worry I have with bumpalo is that I don't think it's been reviewed thoroughly for unsoundness.

Still, bumpalo has been used to great effect in dodrio[1], a React-like library for Rust with really good performance.

[0]: https://crates.io/crates/bumpalo

[1]: https://github.com/fitzgen/dodrio

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#204

Post by pdimitar »

correnos wrote: post #191
pdimitar wrote: post #166

Earlier quoted context omitted.

I am suggesting they apply good practices. I'd never imagine that compilers were actually doing what was stated -- sounds awful. I understand it's tradeoffs and we all have real-world limitations to contend with -- but again, of all the corners that could be cut that's exactly the one I didn't imagine they would. Nasty.

Of the three compilers I've worked on in-depth, only one of them had a "normal" memory management scheme. One of them was unburdened by any thought of freeing stuff, and relied entirely on the application exiting for cleanup. This was very convenient to work with, and never ended up posing an issue. Another used a series of allocation arenas, where certain arenas would be cleared at certain points in the compiler pip…

Judging by the downvotes I am getting (with zero explanation as to why), I'd say that many misunderstood the "good practices" part and took offence, as if I said there's only one good practice.

I was taught -- including at the start of my career when I used exclusively C/C++ (about 18.5y ago) -- to take care of all resources I was using and not rely on runtimes.

I understand and appreciate different usages but to me doing a proper cleanup was the sane default for most programmers. And that's all what I was saying.

Obviously, as one digs deeper in a specialised area where more and more efficiency is demanded then they have to reach for tools that most of us wouldn't normally. That's quite normal and was always interesting for me to read about.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#205

Post by all-fakes »

Reelin wrote: post #54
arcticbull wrote: post #40

Earlier quoted context omitted.

> This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. That doesn't seem to make intuitive sense. A GC has the same problem. A garbage collector has to traverse the data structure…

> The only way to drop an extensively nested structure like this any faster than traversing it would be an arena allocator, and forgetting about the entire arena. Isn't that incompatible with RAII though?

Yes. Some libraries, like typed-arena (and Rust's built-in Vec), will traverse the structure and call the drop implementations, and others, like bumpalo, will just leak the resources.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#206

Post by burpsnard »

masklinn wrote: post #114
tedunangst wrote: post #47

Earlier quoted context omitted.

It would be helpful to see an example from a real application, too.

A very large Vec (say a few million non-empty strings) would do I'd guess, Rust would drop the Vec which would recursively drop each String.

I start thinking about c++ 'placement new' and COBOL copy record structures.

And aquire/release objects from resource pools that manage themselves. And message queue handlers. And pass-by-reference, and object pools..

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#207

Post by ShroudedNight »

MaxBarraclough wrote: post #144
mcguire wrote: post #120

Earlier quoted context omitted.

Unless your destructors do more than deallocation, in which case you will leak whatever other resource you're managing.

A pool can invoke destructors when it is cleared. Might take a bit of overhead (if the pool is to support arbitrary classes), but you could retain the fast pointer-bump allocation.

Not Rust, C++, but in case anybody wants an example: https://github.com/eclipse/omr/commit/fd99ca42fdbed76cc00e68...

TR::Region is the slab allocator used by the JIT in OpenJ9/OMR. The linked commit adds functionality for calling destructors of arbitrary types allocated in the Region.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#208

Post by andrepd »

josephg wrote: post #194
orf wrote: post #86

Earlier quoted context omitted.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

Sure; but when you’re using arenas and things like that you usually you will want different objects allocated into different pools (or with different lifetime properties). Rust only lets you pick one allocator for the entire process, so you can’t specify “all the children of this data structure go in arena A, and this other allocation goes into a traditional heap”. It’s more awkward, but I much prefer Zig’s approach…

>everything that allocates takes the allocator as a parameter

This is also the C++ approach

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#209

Post by burpsnard »

epage wrote: post #10
maxton wrote: post #5

I'm not very familiar with Rust, but I don't understand why you wouldn't just use a reference-to-HeavyThing as the function argument, so that the object isn't moved and then dropped in the `get_size` function?

For these contrived cases, yes, you would just pass a reference to the function but I think the point is to simplify the case down to demonstrate a point.

In the olden days, it was just out.flush(); out.close();

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#210

Post by Reelin »

im3w1l wrote: post #159
Reelin wrote: post #39

Earlier quoted context omitted.

> the ownership system lets you transfer freeing responsibility off-thread safely and cheaply in order to not have it block the critical path This can also trivially be done in other languages. Atomically append your pointer to a queue of "large things that need to be freed" and move on as though you had actually called free. Within a particularly time sensitive loop you can even opt to place pointers into a prealloc…

A lot of C++ code depends on deallocation order for correctness. Like a destructor may want to say bye-bye to a pointed-to-object, and if you reverse order of deallocation, that pointer may be dangling. Consider this code { Window a; ClickHandler* b = new ClickHandler(&a); delete b; } Let's say b tries to deregister itself when it's deleted. This code will work as written. But if you defer the deletion of b, then sta…

I never meant to imply that you could use such tricks without considering the lifetimes of references. Manual memory management already requires being mindful of those though so it's nothing unique to the described situation. Yes Rust offers an advantage here, but it's the same one it offers everywhere else so it doesn't seem relevant to me.

That being said, it seems like most cases where such an approach is worthwhile involve large nested data structures that involve lots of pointer chasing to traverse. All cases I've encountered so far were bulk stores that didn't involve actively interacting with external objects.

Post reply on HN