Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RaptorQ and performance optimization in Rust

cberner.com

Post reply on HN
Print view View original

11–20 of 51 posts

Re: RaptorQ and performance optimization in Rust

#11

Post by MaulingMonkey »

staticassertion wrote: post #10
empath75 wrote: post #2

Anybody want to comment on whether all of his unsafe code was actually necessary? Seems bad for rust that safe code is 25x slower.

SIMD requires unsafe for some reason. Not sure exactly why. I don't see why they used unsafe in `add_assign` - an assert!(octets.len() == other.len()) would likely have elided the bounds checks.

https://users.rust-lang.org/t/how-to-zip-two-slices-efficien... explores eliding the bounds checks even without an assert! (though by using the smaller length as the bounds, which might not be the desired behavior)

Re: RaptorQ and performance optimization in Rust

#12

Post by bb88 »

tgtweak wrote: post #4

Very cool efforts here. Curious what the speed is without those instructions present on the cpu, and what hardware this was run on. The concept of a fountain seems interesting but what is a good use case? Variable strength error correction?

Multicast reliable transmission that scales well to many many clients.

Re: RaptorQ and performance optimization in Rust

#14

Post by herniatedeel »

mmastrac wrote: post #13
layoutIfNeeded wrote: post #6

What is the patent situation with Raptor codes nowadays?

Was just coming to ask the same question. Last I remember fountain codes were pretty locked down.

The inventor of Raptor codes, M. Amin Shokrollahi, sold his company, Digital Fountain, to Qualcomm. Upon the sale to Qualcomm, Qualcomm acquired all of Digital Fountain's IP rights.

Qualcomm has asserted that these Raptor code-related patents (an early one of which was filed in 2004) are standards essential, and require to be licensed from Qualcomm.[1][2]

The below-linked patent would expire in 2024. However, there are a slew of continuation applications that expire much later than 2024.

Update: additionally, there is at least one earlier-dated patent filed in 1999, which expired in February. [3]

[1] https://datatracker.ietf.org/ipr/2554/

[2] https://patents.google.com/patent/US7139960

[3] https://patents.google.com/patent/US6307487

Re: RaptorQ and performance optimization in Rust

#15

Post by sudhirj »

layoutIfNeeded wrote: post #6

What is the patent situation with Raptor codes nowadays?

We have a license and use them for satellite broadcasting, but my understanding is Qualcomm has made statements in https://datatracker.ietf.org/ipr/1511/ that if you use it for a "wireless wide-area standard (for example, a UMTS-compatible handset or Infrastructure equipment)" you'll be charged a standard royalty fee, otherwise they don't care.

Re: RaptorQ and performance optimization in Rust

#16

Post by sudhirj »

tgtweak wrote: post #4

Very cool efforts here. Curious what the speed is without those instructions present on the cpu, and what hardware this was run on. The concept of a fountain seems interesting but what is a good use case? Variable strength error correction?

Let's say you want to send something (say 100KB) to a million listeners, but you don't know which of them is going to be listening when. You'd feed that 100KB into a RaptorQ encoder, configure it to 1KB packets, and it would give you stream of near-infinite 1KB packets that you could broadcast (usually over a satellite, but UDP, multicast, QR codes all work).

Receivers would listen for as many of these packets as they can, as and when they can (the transmission can be "lossy"), and if a receiver pickups up 100 unique packets, any 100 unique packets in any order, it'll have a 99.9% (or something along those lines) chance of decoding the message successfully. Each extra unique packet adds a 9 to the chances.

That's why the "fountain". It's a data fountain, and you can grab any quantity of data off it at any point and still have a good chance of reconstructing the message.

Re: RaptorQ and performance optimization in Rust

#17

Post by sudhirj »

Does anyone know of a good (preferably pure) Go implementation? https://github.com/google/gofountain has mad a good start and implemented the older plain Raptor standard, but I haven't been able to find a good Go library.

If anyone is qualified enough to take a crack at it I can try and arrange for sponsorship as well.

Re: RaptorQ and performance optimization in Rust

#18

Post by detaro »

sudhirj wrote: post #15
layoutIfNeeded wrote: post #6

What is the patent situation with Raptor codes nowadays?

We have a license and use them for satellite broadcasting, but my understanding is Qualcomm has made statements in https://datatracker.ietf.org/ipr/1511/ that if you use it for a "wireless wide-area standard (for example, a UMTS-compatible handset or Infrastructure equipment)" you'll be charged a standard royalty fee, otherwise they don't care.

... as long as you use it for implementing a specific IETF specification. At least that how it reads to me?

Re: RaptorQ and performance optimization in Rust

#19

Post by sudhirj »

detaro wrote: post #18
sudhirj wrote: post #15

Earlier quoted context omitted.

We have a license and use them for satellite broadcasting, but my understanding is Qualcomm has made statements in https://datatracker.ietf.org/ipr/1511/ that if you use it for a "wireless wide-area standard (for example, a UMTS-compatible handset or Infrastructure equipment)" you'll be charged a standard royalty fee, otherwise they don't care.

... as long as you use it for implementing a specific IETF specification. At least that how it reads to me?

Sure, I guess? I don't see why you'd want to deviate from the RFC6330 https://tools.ietf.org/html/rfc6330 - stuff like this is complicated a.f.

Re: RaptorQ and performance optimization in Rust

#20

Post by ignoramous »

minimax wrote: post #3

Can anyone suggest a good book that would serve as an introduction to finite field arithmetic? I keep randomly running into it (e.g. this post), but don't understand it well enough to follow the discussion.

If you would prefer video, then UoCambridge's recording of "Lecture 5: Entropy and Data Compression (IV): Shannon's Source Coding Theorem, Symbol Codes and Arithmetic Coding" is up here: http://videolectures.net/mackay_course_05/

In case you don't find what you're looking for, here are all lectures on "Information Theory, Pattern Recognition, and Neural Networks": http://videolectures.net/course_information_theory_pattern_r...

I'd be remiss not to mention that the lectures are by the late David MacKay: https://news.ycombinator.com/item?id=11500221

Post reply on HN