Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

191–200 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#191

Post by correnos »

pdimitar wrote: post #166
qzw wrote: post #132

Earlier quoted context omitted.

Well, then that’s not the original use case anymore, and it’ll have to be re-engineered. In the meantime it may have been used for years and the perf difference may have saved many developer-years collectively across its user base. Surely you’re not suggesting that the compiler developers should be prematurely optimizing for future use cases that they may not even have envisioned.

I am suggesting they apply good practices. I'd never imagine that compilers were actually doing what was stated -- sounds awful. I understand it's tradeoffs and we all have real-world limitations to contend with -- but again, of all the corners that could be cut that's exactly the one I didn't imagine they would. Nasty.

Of the three compilers I've worked on in-depth, only one of them had a "normal" memory management scheme.

One of them was unburdened by any thought of freeing stuff, and relied entirely on the application exiting for cleanup. This was very convenient to work with, and never ended up posing an issue.

Another used a series of allocation arenas, where certain arenas would be cleared at certain points in the compiler pipeline. This made for both speedy alloc/freeing and avoided leaks, since you weren't at risk of "forgetting" a data structure. It was also a major headache to keep track of exactly what the longest lifetime of a long-lived datastructure might be, and to pick an arena that won't be cleared in the meantime. Unfortunately the programs compiled with this compiler were large enough that we certainly couldn't have gotten away with just leaking memory; we sometimes OOMed as-is!

The third used standard C++ memory management. This compiler was quite simple, and the vast majority of its data used stack-based lifetimes. For a more complex compiler this would've become a headache.

I think that all of these compilers chose the correct allocation strategy for what they were doing. "Good practices" aren't as universal as we might like to believe, they depend entirely on the context in which a tool is designed to operate. And yes, we can guard to some extent against that context changing, but for the most part that's why we keep getting paid.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#192

Post by viraptor »

jpitz wrote: post #190
rubber_duck wrote: post #140

Earlier quoted context omitted.

Avoiding leaks is not optimisation, it's a matter of correctness - not freeing memory is an optimisation based on a very shortsighted assumption that is not practical for any new language (modern languages are expected to come with language server support)

Correctness means adherence to the spec, not some contrived absolute truth.

But the spec usually has some implicit assumptions. Usually it's "app doesn't leak memory" in the same way nobody explicitly specifies "result of an addition of natural numbers should match ...".

We don't go around saying "oh, you didn't want modulo 5 arithmetic? You should've put that in the spec, not rely on some contrived absolute truth".

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#193

Post by francasso »

orf wrote: post #86
papaf wrote: post #79

Earlier quoted context omitted.

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

I don't get this. Usually I either don't care about how stuff is allocated, or when I care I want to be specific about what allocator should be used where. In the second situation just setting a global allocator wouldn't cut it most of the time, I would want support for customization when I instantiate a data structure.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#194

Post by josephg »

orf wrote: post #86
papaf wrote: post #79

Earlier quoted context omitted.

This deallocation trick is neat but in C and C++ you could use a memory pool to do this. In theory, you could also use a memory pool in Rust but I think the standard library uses malloc without some way of overriding this behaviour.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

Sure; but when you’re using arenas and things like that you usually you will want different objects allocated into different pools (or with different lifetime properties). Rust only lets you pick one allocator for the entire process, so you can’t specify “all the children of this data structure go in arena A, and this other allocation goes into a traditional heap”.

It’s more awkward, but I much prefer Zig’s approach here where everything that allocates takes an allocator as a parameter. Usually the allocator is specified at compile time - in which case zig can generate identical code to the rust compiler. But when you want the flexibility, it’s there.

Aside from compilers, this is heavily used in video games where there’s often a lot of objects that get allocated per frame, and can be discarded all together. And in that case rust’s lifetime tracking would be a huge asset. The dovecot email server (C) also makes superb use of a mix of memory containers for different tasks. Given how messy email parsing is, dovecot is an absolute pleasure to read.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#195

Post by jpitz »

viraptor wrote: post #192
jpitz wrote: post #190

Earlier quoted context omitted.

Correctness means adherence to the spec, not some contrived absolute truth.

But the spec usually has some implicit assumptions. Usually it's "app doesn't leak memory" in the same way nobody explicitly specifies "result of an addition of natural numbers should match ...". We don't go around saying "oh, you didn't want modulo 5 arithmetic? You should've put that in the spec, not rely on some contrived absolute truth".

Okay, but we're talking about an application, a piece of software who's primary intention is to run for a short period of time, parse a text file and transform it. It will be ran many many many times a day by developers whose time is expensive. Language servers are a very new concept in terms of being part of the day-to-day tool chains for most developers. Trading garbage collection for compile time was absolutely a winning strategy for compiler writers. It made them money.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#196

Post by setr »

pdimitar wrote: post #172
rcxdude wrote: post #170

Earlier quoted context omitted.

Deallocation at the end of a program's execution can substantially add to its runtime, and it's entirely waste. It's a much more common strategy than you might think.

You are right, I indeed didn't know it was that common. But still, in a world where languages and runtimes are also judged by their ability to run in lambda/serverless setups, I'd think this practice will start being obsolete, wouldn't you think? (What I mean is that I imagine that any serverless function that runs in severely constrained and measured environments like the AWS Lambda would gain a significant edge ove…

I don't understand; a lambda function executes and closes, and any leak gets handled by lambda runtime (by freeing the whole lambda function)-- exactly the same as running a cli tool. It seems to me that the server-less context is actually exactly where you'd employ this strategy: the programs by definition cannot be long-lived, so unless they generate so much garbage as to oom in that timeline, there is no need to deallocate anything yourself.

The leak to watch out for is reduced to the persisted data (eg in S3) between executions

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#197

Post by burpsnard »

loufe wrote: post #37
chowells wrote: post #31

This is the standard problem with tracing data structures to free them. You frequently run into it with systems based on malloc/free or reference counting. The underlying problem is that freeing the structure takes time proportional to the number of pointers in the structure it has to chase. Generational/compacting GC has the opposite problem. Garbage collection takes time proportional to the live set, and the amount…

I've not worked with any language thus far without automatic garbage collecting, so this was definitely a neat read for me. It sounds rather elegant.

It's worth popping the hood and getting your fingers dirty. C was written in an era where memory was a scarce and precious resource to be grudgingly used if absolutely necessary

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#198

Post by steveklabnik »

josephg wrote: post #194
orf wrote: post #86

Earlier quoted context omitted.

You can change the global allocator in any rust project. You can write your own easy enough, or use one like jemalloc

Sure; but when you’re using arenas and things like that you usually you will want different objects allocated into different pools (or with different lifetime properties). Rust only lets you pick one allocator for the entire process, so you can’t specify “all the children of this data structure go in arena A, and this other allocation goes into a traditional heap”. It’s more awkward, but I much prefer Zig’s approach…

Not in an abstract way, but you can do this. The compiler has (had? It’s been a while) two different major arenas, in my understanding. There’s just no non-global allocator trait yet, so it’s not easy to abstract over.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#199

Post by jashmatthews »

viraptor wrote: post #192
jpitz wrote: post #190

Earlier quoted context omitted.

Correctness means adherence to the spec, not some contrived absolute truth.

But the spec usually has some implicit assumptions. Usually it's "app doesn't leak memory" in the same way nobody explicitly specifies "result of an addition of natural numbers should match ...". We don't go around saying "oh, you didn't want modulo 5 arithmetic? You should've put that in the spec, not rely on some contrived absolute truth".

It’s usually the opposite! Correctness work usually has the implicit assumption of infinite memory.
Post reply on HN