Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust: Dropping heavy things in another thread can make your code 10000x faster

abramov.io

Post reply on HN
Print view View original

271–280 of 285 posts

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#271

Post by MaxBarraclough »

ShroudedNight wrote: post #207
MaxBarraclough wrote: post #144

Earlier quoted context omitted.

A pool can invoke destructors when it is cleared. Might take a bit of overhead (if the pool is to support arbitrary classes), but you could retain the fast pointer-bump allocation.

Not Rust, C++, but in case anybody wants an example: https://github.com/eclipse/omr/commit/fd99ca42fdbed76cc00e68... TR::Region is the slab allocator used by the JIT in OpenJ9/OMR. The linked commit adds functionality for calling destructors of arbitrary types allocated in the Region.

Interesting, thanks. Seems to require use of a common base class though, i.e. to use a class with this pool, you have to inherit from Destructable, or else create a subclass that does. Seems ugly. Perhaps that's the only way it can be done in C++ though.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#272

Post by pshc »

tsimionescu wrote: post #260
pshc wrote: post #244

Earlier quoted context omitted.

Yes it’ll reduce latency, but doesn’t it also increase parallelism? A single-threaded program ought to improve overall, unless the extra overhead you mentioned dominates. A parallel program might improve or not. I think if you wanted to do deferred destruction right, ideally you’d mod an allocator to have functions like (alloc_local, alloc_global, free_now, free_deferred) to avoid exhausting memory. Traits could make…

> Also I admit I don’t understand why “you won’t have any backpressure on your allocations,” shouldn’t deferred destruction give you more backpressure if anything? I am probably confused. I think the point is that, if the same thread is doing both allocation and de-allocation, the thread is naturally prevented from allocating too much by the work it must do to de-allocate. If you move the de-allocation to another thr…

Ah, I see what you mean, thanks!

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#273

Post by Reelin »

tsimionescu wrote: post #258
Reelin wrote: post #88

Earlier quoted context omitted.

My understanding is that finalizers are special cased for a generational GC, have a tendency to introduce significant overhead, and (as with any GC scheme) generally run at unpredictable times. My impression is that RAII idioms are strongly discouraged in conjunction with most GC ecosystems.

Actually, RAII idioms are generally encouraged, but finalizers are not. For the rare few classes that hold resources in GC languages, the Resource is usually Acquired at Initialization as well, and released in a special method (IDisposable.Dispose() in C#, AutoCloseable.Close() in Java). There is also usually some special syntax for automatically calling this syntax when the object exits some scope, though that scope…

This is a really good point. I had completely overlooked various deterministic resource cleanup idioms as being equivalent to RAII in all but name.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#274

Post by AaronFriel »

jkoudys wrote: post #176

It'd be interesting to implement this on a type that would defer all of these drop threads (or one big drop threads built off a bunch of futures) until the end of some major action, like sending the http response on an actix-web thread. Could be a great way to get the fastest possible response time, since then the client has their response before any delay on cleanup.

There is no such thing as a free lunch here, so it would reduce the unloaded response time but should have no effect (or a negative impact) on a highly loaded server's response time. I'm finding this out when benchmarking a message passing/queue management system. Anything I do to defer work onto a separate threadpool improves latency up to a point, then reduces throughput.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#275

Post by GuB-42 »

fpgaminer wrote: post #61

Some important things I think people should note before blindly commenting: * The example code is obviously contrived. The real gist is that massive deallocations in the UI thread cause lag, which the example code proves. That very thing can easily happen in the real world. * I didn't see any difference on my machine between a debug build and a release build. * The example is preforming 1 _million_ deallocations. Tha…

I don't know if Rust can do it (unsafe?) but in C and C++, I sometimes end up writing a custom allocator. It is often one of the most significant optimizations.

For example, I had the "million strings" problem once, literally millions. The solution was to put every string into a single large buffer and the pointers in another buffer. Not only I could deallocate everything at once but I also saved a bit of RAM by not aligning (not needed for strings).

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#276

Post by jkoudys »

AaronFriel wrote: post #274
jkoudys wrote: post #176

It'd be interesting to implement this on a type that would defer all of these drop threads (or one big drop threads built off a bunch of futures) until the end of some major action, like sending the http response on an actix-web thread. Could be a great way to get the fastest possible response time, since then the client has their response before any delay on cleanup.

There is no such thing as a free lunch here, so it would reduce the unloaded response time but should have no effect (or a negative impact) on a highly loaded server's response time. I'm finding this out when benchmarking a message passing/queue management system. Anything I do to defer work onto a separate threadpool improves latency up to a point, then reduces throughput.

If you're bottlenecked, then certainly. There's no free lunch, but for us, problems that can be solved by simply scaling up the resources on the host as relatively cheap as free vs expensive developer time. When we're purely focused on sales and not anywhere close to hitting a full mem/cpu bottleneck, this would bee good.

This situation you describe sounds a lot like dealing with garbage-collection cycles, so you give a good recommendation on something to watch out for, as rust performing at the level of a GC'd language removes a big reason for choosing rust.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#277

Post by pjmlp »

jpitz wrote: post #195
viraptor wrote: post #192

Earlier quoted context omitted.

But the spec usually has some implicit assumptions. Usually it's "app doesn't leak memory" in the same way nobody explicitly specifies "result of an addition of natural numbers should match ...". We don't go around saying "oh, you didn't want modulo 5 arithmetic? You should've put that in the spec, not rely on some contrived absolute truth".

Okay, but we're talking about an application, a piece of software who's primary intention is to run for a short period of time, parse a text file and transform it. It will be ran many many many times a day by developers whose time is expensive. Language servers are a very new concept in terms of being part of the day-to-day tool chains for most developers. Trading garbage collection for compile time was absolutely a…

Language servers go back all the way to Xerox PARC workstations and Lisp machines.

C++ had its very first language servers via Lucid's Energize C++ and Visual Age for C++ v4.

Here is the 1993 video and related paper from Lucid.

https://www.youtube.com/watch?v=pQQTScuApWk

http://www.dreamsongs.com/Cadillac.html

And some information regarding VA, unfortunately most is missing from online world.

https://www.ecomstation.it/pido2/home/esterni/vac40os2.pdf (codestore)

http://www.edm2.com/index.php/VisualAge_C%2B%2B_4.0_Review

It is also why Delphi, VB and C++ Builder were already such a pleasure to use versus the Makefile and vi world of UNIX.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#278

Post by pjmlp »

ahupp wrote: post #160
justinpombrio wrote: post #129

Earlier quoted context omitted.

EDIT: See kevincox's reply. Rust will bitwise copy the containing type , which is typically very cheap. For example, it you move a String, it will copy the String struct, which contains a couple pointers and a length (or something along those lines). Importantly, it will not copy the underlying char array. I was thinking of the following code, where I believe the assignment to y is actually free. Though apparently th…

That's not exactly true. In C++ terms, the example code is moving a value: std::map foo; someFunction(std::move(foo)); And not moving a pointer like: std::unique_ptr > foo = ...; someFunction(std::move(foo)); So it copies sizeof(std::map ), not a pointer.

Not necessary, std::map::map(&&) is free to move just the internal pointers and bookkeeping data, only dumb implementations would move the memory blocks where the map data is actually stored, as per ISO C++ requirements.

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#279

Post by jfkebwjsbx »

zucker42 wrote: post #270
jfkebwjsbx wrote: post #229

Earlier quoted context omitted.

> after all the main point of, for example, destructors in C++ is to make sure you don't leak memory, which the GC solves No, C++ destructors are used for finalizers, not memory management. Memory deallocation is a particular use case for finalizers which is avoided when performance is a concern. > Another point is the C++/Rust pattern of each object recursively freeing the objects it owns presumably leads to slower…

> No, C++ destructors are used for finalizers, not memory management. In the original comment I may have overstated this. I was ignoring the other uses of destructors because the context of the discussion was memory management. But memory management is a huge portion of what destructors do in C++. Consider a vector of strings (`vector `). The destructor deallocates the memory for all the strings, then deallocates the…

> Consider a vector of strings (`vector`). The destructor deallocates the memory for all the strings, then deallocates the memory for the vector.

A vector (or binary tree as you mentioned later) of heap allocated elements in a performance critical path is a likely candidate to be redesigned if possible.

> As far as I can tell, you either have to pointer chase or use a custom allocation strategy. At some point with the second option, you're basically creating an ad-hoc garbage collector.

A custom, ad-hoc strategy is the way to go if you need them dynamically allocated and performance at the same time, yeah.

I wouldn't call that a GC, though. A GC usually refers to a global solution.

> But I think we may be in vigorous agreement here

Sounds like it!

Re: Rust: Dropping heavy things in another thread can make your code 10000x faster

#280

Post by crimsonalucard1 »

I guess choosing when or how a program deallocates is important in a language that's close to the metal.

Rust tries to be zero cost while providing abstractions that make it seemingly a high level language but ultimately things like this show that it's not exactly zero cost because abstractions can incur hidden penalties. There needs to be some internal syntax that allows a rust user to explicitly control deallocation when needed.

If I started reading code where people would randomly move a value into another thread and essentially do nothing I would be extremely confused. Any language that begins to rely on "trick" or "hacks" as standard patterns exposes a design flaw.

Maybe if rust provided special syntax that a function can be decorated with so that it does deallocation in another thread automatically? Or maybe an internal function called drop_async...? This would make this pattern an explicit part of the language rather than a strange hack/trick.

Post reply on HN