Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dav2d

jbkempf.com

Post reply on HN
Print view View original

131–140 of 212 posts

Re: Dav2d

#131

Post by cogman10 »

dcsommer wrote: post #128
cogman10 wrote: post #118

Earlier quoted context omitted.

> As rav1d shows rav1d is not a full rewrite of dav1d to rust. So it really doesn't show that. It's currently C + rust + asm. I don't think we can say anything about what this does or does not prove about the performance of safe code. > Performance should not be priority #1. Security should be. Entirely depends on the application. The reason rust has `unsafe` is because there's some situations where performance needs…

Codecs are difficult and expensive to develop. Therefore they get reused in many contexts, including security critical ones. Sandboxing is shown over and over to not be a great security solution, so what this means in practice is that security-critical software that needs software decoding get pwned because software engineers don't care to prioritize it in the first place. Why shouldn't safety be the default? If you…

> Why shouldn't safety be the default?

Because safe code isn't fast enough to decode live video.

> If you really want to, it wouldn't be too hard to maintain a patch on top of rustc to drop the bounds checks if you want to compile object files without them.

Yeah, but then you are undermining safety in a critical way that does lead to security vulnerabilities (buffer overflow). And you are also now maintaining and requiring other devs for a project to use a custom version of rustc. That's certainly part of the reason that's simply not happened.

But another major part of it is that encoders end up with a lot of custom ASM regardless. That custom ASM is going to be where vulnerabilities end up. You don't really escape that by using rust.

If you are already abandoning where you critically need safety the most for performance, then why pick a language that additionally penalizes you for using unsafe constructs?

> Software decoding has a safety culture problem, and we need to talk about it.

Compilers and languages have an optimization problem that we need to talk about. SIMD optimizations remain a very hard thing for compilers to get right. We should talk about what it'd take to make compilers better and the reasons for why codec devs need to drop down to asm instead of using a high level compiler.

There might not be a solution to this problem, there are reasons for it.

Re: Dav2d

#132

Post by ihattendorf »

thrownthatway wrote: post #110
halJordan wrote: post #85

Earlier quoted context omitted.

Well yes? The platforms only accept certain resolution/bitrates and also most of America isnt running 1gig up. They're running 5-30 mbps up. So yeah they need to encode it.

> They're running 5-30 mbps up Do you not have 98% high speed 5G coverage?

Data caps make that hard. While everyone likes to claim unlimited data, I'm not aware of any providers that don't have a heavy data user clause where they'll deprioritize your data if you're a top ~5% data user (usually somewhere over ~1TB/month).

You also will need _some_ sort of encoding locally before uploading, even if it's minimal, which could lead to issues when encoded again (although there are codecs available to minimize this).

Re: Dav2d

#135

Post by Forgeties79 »

esposm03 wrote: post #127
thrownthatway wrote: post #84

Earlier quoted context omitted.

Is there a compelling reason encoding needs to be done locally? The point of encoding is to reduce downstream bandwidth for the viewer, and upstream bandwidth for the distribution network. The content creator only needs to upload it once.

Yes. An uncompressed 1080p, 60fps video with 24-bit color depth would need around 3Gbps to be streamed. And even if you don't need to stream it, that would still consume a sizeable portion of the write throughput of the fastest SSDs currently available; if you go up to 4K, you'd actually exceed that by a lot (not to mention, 1tb of storage would last for about 10 minutes of video).

Who is regularly watching uncompressed videos outside of production environments? That’s got to be a very small population.

Re: Dav2d

#136

Post by skelpmargyar »

IshKebab wrote: post #95
poly2it wrote: post #7

Sorry if this sounds naive, but does it make sense to write a codec library in C/ASM considering how well Rust is progressing, especially when, as the author puts it, AV2 decoding is roughly five times more complex than AV1 decoding ?

I don't know why you've been down-voted. It definitely isn't an optimal decision. A video codec isn't all assembly. There's plenty of plain unsafe C code. E.g. this is the first random file I clicked. It has a ton of raw C pointer stuff just begging to be exploited. https://code.videolan.org/videolan/dav2d/-/blob/main/src/dat... There is a project to write an AV1 decoder in Rust: Rav1d (really stretching the name her…

Of course any random C file is going to have pointers. Where can anything in the linked code be exploited? It seems like they're testing for bad input data with asserts to catch bugs in some functions, and properly validating bad inputs in others. Just because they're writing C doesn't mean it's vulnerable.

How can you claim nobody cares about 5%? A 5% performance increase is significant. And video decoding is not always for playback, where 5% may not matter as much.

Re: Dav2d

#137

Post by breve »

pantalaimon wrote: post #113

I'm not quite convinced a 25% reduction in size is worth effectively obsoleting all devices that have hardware decoders for AV1 but will struggle to decode AV2

When you host videos with near 17 billion views you're going to want to stream those videos in as few bits as possible:

https://www.youtube.com/watch?v=XqZsoesa55w

That extra 25% becomes worth it.

Nothing will become obsolete. AV1 will stick around for a long time. And YouTube still does H.264 encodes to support old devices.

Re: Dav2d

#138

Post by Anduia »

NorwegianDude wrote: post #109
BetterThanSober wrote: post #80

Earlier quoted context omitted.

I don't know if I'm underestimating HN's reach but I doubt we did that, probably traffic from a much bigger aggregator/forum

Hacker news doesn't generate much traffic, despite what people are saying. The host here has a limit of 160000 files served each day. That is extremely low. If the site has an icon, css, a js file and a few images it's 10 files each visit. That's will limit it to 16k visits/day. If there are more files loaded it might just handle a few thousand visits, and they have received more than that from HN now.

Well, if every asset request hits the origin (no CDN, no caching...) that would be a misconfiguration of the website. It should never happen for a blog.

Re: Dav2d

#140

Post by adgjlsfhk1 »

IshKebab wrote: post #93
WD-42 wrote: post #81

Earlier quoted context omitted.

dav1d is the av1 decoder and it’s an insane feat of engineering. Written in assembly, it even eschews the normal c calling convention to get even better performance.

The normal C calling convention is really only for cross-binary calls (e.g. between shared libraries). If you're not doing that you can ignore it; it's not a weird thing to do. It would be odd to strictly follow it in assembly and I assume compilers don't either.

Unfortunately, in absence of inlining, compilers mostly respect calling convention even when they don't have to.
Post reply on HN