RaptorQ and performance optimization in Rust
cberner.com
RaptorQ and performance optimization in Rust
1–10 of 51 posts
Re: RaptorQ and performance optimization in Rust
#2Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.
Re: RaptorQ and performance optimization in Rust
#3Can anyone suggest a good book that would serve as an introduction to finite field arithmetic? I keep randomly running into it (e.g. this post), but don't understand it well enough to follow the discussion.
Re: RaptorQ and performance optimization in Rust
#4Very cool efforts here. Curious what the speed is without those instructions present on the cpu, and what hardware this was run on.
The concept of a fountain seems interesting but what is a good use case? Variable strength error correction?
Re: RaptorQ and performance optimization in Rust
#5Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.
Anything outside of rust (like assembly or simd) is not safe-ensurable by rusts build-in, just like interfacing with c libraries in golang is not safe.
I would imagine without these simd optimizations you would see a 2X slowdown at least.
Re: RaptorQ and performance optimization in Rust
#6What is the patent situation with Raptor codes nowadays?
Re: RaptorQ and performance optimization in Rust
#7Can anyone suggest a good book that would serve as an introduction to finite field arithmetic? I keep randomly running into it (e.g. this post), but don't understand it well enough to follow the discussion.
Huffman: Fundamentals of Error-Correcting Codes
ISBN 9780521782807
Re: RaptorQ and performance optimization in Rust
#8Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.
[deleted]
Re: RaptorQ and performance optimization in Rust
#9Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.
> Seems bad for rust that safe code is 25x slower.
Most of the optimizations listed have nothing to do with unsafe, and have to do with algorithmic changes that you could apply in any language, and could apply to safe code just as well as to unsafe code. Even if we pretend all unsafe used there is necessary, and interpret the source in the worst possible light for rust - that the optimization steps would be impossible without unsafe - I think we'd only be talking 4x to 5x slower from this article, not 25x.
But - I strongly suspect not all that unsafe is necessary, and the post certainly doesn't claim it is.
The first code snippet uses unsafe to skip bounds checks. There are plenty of other ways to get the optimizer to skip them for you in safe code - or at the very least, hoist them out of the loop instead. This seems like a good read showing some concrete examples:
https://coaxion.net/blog/2018/01/speeding-up-rgb-to-grayscal...
Explicitly using SIMD code like the second code snippet does might require unsafe for the transmuted slicing under the hood (implemented as raw pointer derefs in this case), but it's possible to build safe abstractions on top of that. That blog post links this, for example, although it didn't end up using it (EDIT: Actually, it initially didn't, but now does): https://github.com/AdamNiederer/faster . It does use unsafe internally: https://github.com/AdamNiederer/faster/search?q=unsafe&unsco...
Re: RaptorQ and performance optimization in Rust
#10Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.
SIMD requires unsafe for some reason. Not sure exactly why.
I don't see why they used unsafe in `add_assign` - an assert!(octets.len() == other.len()) would likely have elided the bounds checks.