Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dav2d

jbkempf.com

Post reply on HN
Print view View original

171–180 of 212 posts

Re: Dav2d

#171

Post by alkonaut »

Arodex wrote: post #22
poly2it wrote: post #7

Sorry if this sounds naive, but does it make sense to write a codec library in C/ASM considering how well Rust is progressing, especially when, as the author puts it, AV2 decoding is roughly five times more complex than AV1 decoding ?

The algorithms deployed in these kind of codecs take into account not only human vision and mathematical laws of information, but also nitty-gritty details of how computers work, which are optimally exploited by directly having humans write detailed assembly rather than a compiler make a best guess and effort.

Surely 100% of these low level features are availale in rust too? I understand it is a massive undertaking and builds off the previous codec(s) but writing these things by hand such as inline assembly seems to be as easy if not easier in Rust?

And as soon as you walk into concurrency territory for a complex codec like this then it seems almost impossible for humans to do correctly while retaining safety.

Re: Dav2d

#172

Post by alkonaut »

jbk wrote: post #16
poly2it wrote: post #7

Sorry if this sounds naive, but does it make sense to write a codec library in C/ASM considering how well Rust is progressing, especially when, as the author puts it, AV2 decoding is roughly five times more complex than AV1 decoding ?

Because it's 5 times more complex, you need to get the maximum performance available. Therefore more ASM than ever. Rust does not bring more performance. Just more safety.

It brings tooling that is a LOT easier. Just things like dependency management, test running and so on is so much better in Rust than in C, even if you happen to write the exact same code because you basically write unsafe code and hand rolled assembly for many things. I think this is people using the tool they know rather than the best tool (And if you know a tool well, it might become the best tool for the job because of that). It could be because a huge chunk of existing code can be re-used. But all else being equal (existing code, existing developers don't exist) I refuse to believe a codec should ever be written in C ever again.

Re: Dav2d

#173

Post by gblargg »

Forgeties79 wrote: post #135
esposm03 wrote: post #127

Earlier quoted context omitted.

Yes. An uncompressed 1080p, 60fps video with 24-bit color depth would need around 3Gbps to be streamed. And even if you don't need to stream it, that would still consume a sizeable portion of the write throughput of the fastest SSDs currently available; if you go up to 4K, you'd actually exceed that by a lot (not to mention, 1tb of storage would last for about 10 minutes of video).

Who is regularly watching uncompressed videos outside of production environments? That’s got to be a very small population.

The context was remotely encoding the video, which would require sending the uncompressed stream.

Re: Dav2d

#174

Post by gblargg »

Ecco wrote: post #130
esposm03 wrote: post #127

Earlier quoted context omitted.

Yes. An uncompressed 1080p, 60fps video with 24-bit color depth would need around 3Gbps to be streamed. And even if you don't need to stream it, that would still consume a sizeable portion of the write throughput of the fastest SSDs currently available; if you go up to 4K, you'd actually exceed that by a lot (not to mention, 1tb of storage would last for about 10 minutes of video).

Using raw uncompressed bitrate is a bit disingenuous. How about comparing an older, widely supported codec like H.264 as a baseline?

If you compressed it with H.264, it wouldn't make much sense to send it remotely to be encoded with a better codec.

Re: Dav2d

#175

Post by imtringued »

dcsommer wrote: post #128
cogman10 wrote: post #118

Earlier quoted context omitted.

> As rav1d shows rav1d is not a full rewrite of dav1d to rust. So it really doesn't show that. It's currently C + rust + asm. I don't think we can say anything about what this does or does not prove about the performance of safe code. > Performance should not be priority #1. Security should be. Entirely depends on the application. The reason rust has `unsafe` is because there's some situations where performance needs…

Codecs are difficult and expensive to develop. Therefore they get reused in many contexts, including security critical ones. Sandboxing is shown over and over to not be a great security solution, so what this means in practice is that security-critical software that needs software decoding get pwned because software engineers don't care to prioritize it in the first place. Why shouldn't safety be the default? If you…

What's supposed to be the big source of unsafety in codecs though? Feels like the problem here is that C developers are ruining the reputation of C with their garbage code.

Bounds checking as a source of slowdown is overrated in a niche where you're working on fixed size blocks. It feels like the C developers are getting the parts outside the ASM kernels wrong.

Re: Dav2d

#176

Post by Forgeties79 »

thrownthatway wrote: post #110
halJordan wrote: post #85

Earlier quoted context omitted.

Well yes? The platforms only accept certain resolution/bitrates and also most of America isnt running 1gig up. They're running 5-30 mbps up. So yeah they need to encode it.

> They're running 5-30 mbps up Do you not have 98% high speed 5G coverage?

Just over half the world’s smartphone users do (meaning almost half don’t), and certain countries/areas have way more coverage than others. And a massive number of people have limited data per month, which means it’s also a cost concern.

Leaner delivery is not just ethical, but it also makes better business sense.

Re: Dav2d

#177

Post by asimovDev »

MaxikCZ wrote: post #35
UnlockedSecrets wrote: post #12

Earlier quoted context omitted.

Return of the 8MB Shrek encodes?

https://web.archive.org/web/20210416200451/https://cdn.disco... Shrek 1 at 8.34MB including audio.. insane

I once watched an entire movie (around 90 mins) on a Nokia 6303. This reminds me of that

Re: Dav2d

#178

Post by rafaelmn »

alkonaut wrote: post #171
Arodex wrote: post #22

Earlier quoted context omitted.

The algorithms deployed in these kind of codecs take into account not only human vision and mathematical laws of information, but also nitty-gritty details of how computers work, which are optimally exploited by directly having humans write detailed assembly rather than a compiler make a best guess and effort.

Surely 100% of these low level features are availale in rust too? I understand it is a massive undertaking and builds off the previous codec(s) but writing these things by hand such as inline assembly seems to be as easy if not easier in Rust? And as soon as you walk into concurrency territory for a complex codec like this then it seems almost impossible for humans to do correctly while retaining safety.

Why ? If it's shared reads and scoped writes (read-only look up, output to a thread owned buffer span) concurrency seems pretty straightforward.

Rust can only prove a limited subset of correct programs to be safe, when you're doing bare metal stuff you've often not in that subsystem and drop down to unsafe. I'm guessing there's always stuff that's not perf critical and can live in Rust sandbox - so not saying no wins - but it doesn't sound like Rust is a no-brainer.

Re: Dav2d

#179

Post by BetterThanSober »

NorwegianDude wrote: post #109
BetterThanSober wrote: post #80

Earlier quoted context omitted.

I don't know if I'm underestimating HN's reach but I doubt we did that, probably traffic from a much bigger aggregator/forum

Hacker news doesn't generate much traffic, despite what people are saying. The host here has a limit of 160000 files served each day. That is extremely low. If the site has an icon, css, a js file and a few images it's 10 files each visit. That's will limit it to 16k visits/day. If there are more files loaded it might just handle a few thousand visits, and they have received more than that from HN now.

Yeah misconfig would also be my guess, too much cache miss
Post reply on HN