Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dav2d

jbkempf.com

Post reply on HN
Print view View original

121–130 of 212 posts

Re: Dav2d

#121

Post by ethin »

NorwegianDude wrote: post #109
BetterThanSober wrote: post #80

Earlier quoted context omitted.

I don't know if I'm underestimating HN's reach but I doubt we did that, probably traffic from a much bigger aggregator/forum

Hacker news doesn't generate much traffic, despite what people are saying. The host here has a limit of 160000 files served each day. That is extremely low. If the site has an icon, css, a js file and a few images it's 10 files each visit. That's will limit it to 16k visits/day. If there are more files loaded it might just handle a few thousand visits, and they have received more than that from HN now.

Wait really? I'm not really sure what to think and I posted before I saw this... I wonder why the limit is so low?

Re: Dav2d

#122

Post by Const-me »

dcsommer wrote: post #107
cogman10 wrote: post #32

Earlier quoted context omitted.

Encoder and decoder writers frequently need extremely fine grain control over SIMD instructions in order to get good performance. The way they weave these instructions can be very hard to express with a high level language. Further, there's a ton of work with arrays and importantly parts of arrays. They can, for example, need to extract every other element up to 1/2 the array. Unfortunately, rust has runtime array bo…

Performance should not be priority #1. Security should be. Why do we slow down all CPUs to prevent SPECTRE attacks yet continue to write in C? As rav1d shows, the perf loss is far less to migrate from C to Rust than it is to apply SPECTRE mitigations, and adding a sandbox around a memory-unsafe codec is going to be way more expensive again than using Rust code to start.

> Performance should not be priority #1. Security should be.

For a web browser, or a server in a bank, sure. For anything else, questionable.

> adding a sandbox around a memory-unsafe codec is going to be way more expensive

In modern world, overhead of strong sandboxes is surprisingly small. A nuclear but most reliable option is hardware assisted VM. On modern computers with SLAT and virtualized IO the overhead for most use cases is negligible. If you want something lighter weight, can use a multi-user nature of all modern OS kernels and isolate into a separate process with restricted permissions. Sandboxing overhead is approximately zero.

Re: Dav2d

#123

Post by F3nd0 »

kiicia wrote: post #115
Caspy7 wrote: post #75

Earlier quoted context omitted.

He only mentioned decode complexity. Would be interesting to know the average encode complexity compared to AV1.

Encoding speed even on Mac Studio is atrocious, it’s in range of single frames per second as opposed to realtime+ for even h265

The specification for AV2 has only been finalised very recently, so performant encoders have not yet been developed. Meaningful comparison to older codecs like H265 and AV1 will only be possible once that has changed. (It'll be slower, but almost certainly not one-frame-per-second slow.)

Re: Dav2d

#124

Post by F3nd0 »

the__alchemist wrote: post #43

Not to be confused with Da4vid (world-class hacker and owner of the Black sun) or D4vd (rap artist and alleged murderer)

I don’t think the chances of confusion between a niche celebrity and a video decoder are very big.

Re: Dav2d

#125

Post by nick__m »

staindk wrote: post #48
the__alchemist wrote: post #43

Not to be confused with Da4vid (world-class hacker and owner of the Black sun) or D4vd (rap artist and alleged murderer)

Or Dave2D, popular tech youtuber

This thread made me think of this song/sketch from Kids in the Hall.

https://www.youtube.com/watch?v=dEGCdVADLWY

Re: Dav2d

#126

Post by nick__m »

throawayonthe wrote: post #74
poly2it wrote: post #7

Sorry if this sounds naive, but does it make sense to write a codec library in C/ASM considering how well Rust is progressing, especially when, as the author puts it, AV2 decoding is roughly five times more complex than AV1 decoding ?

yes it makes sense to use C/ASM here, but if you're curious, there is a rust port of dav1d named rav1d: https://github.com/memorysafety/rav1d it's not much slower than the original C/ASM implementation (last i checked ~5%?) but that matters here

It's a Rust/ASM port, look there: https://github.com/memorysafety/rav1d/blob/main/src/ext/x86/...

I am not sure if it is that much safer than the C version when raw assembly is still required.

Re: Dav2d

#127

Post by esposm03 »

thrownthatway wrote: post #84
kmfrk wrote: post #60

Earlier quoted context omitted.

Intel's Arc dGPUs were really compelling for dedicated AV1 encode and decode, especially the small form factor of some cards. You could even fit it as a secondary card in a PC dedicated to recording and encode workflows for OBS. Hope we get a similar option with future lineups that support AV2, especially given how popular video creation and streaming are now.

Is there a compelling reason encoding needs to be done locally? The point of encoding is to reduce downstream bandwidth for the viewer, and upstream bandwidth for the distribution network. The content creator only needs to upload it once.

Yes.

An uncompressed 1080p, 60fps video with 24-bit color depth would need around 3Gbps to be streamed. And even if you don't need to stream it, that would still consume a sizeable portion of the write throughput of the fastest SSDs currently available; if you go up to 4K, you'd actually exceed that by a lot (not to mention, 1tb of storage would last for about 10 minutes of video).

Re: Dav2d

#128

Post by dcsommer »

cogman10 wrote: post #118
dcsommer wrote: post #107

Earlier quoted context omitted.

Performance should not be priority #1. Security should be. Why do we slow down all CPUs to prevent SPECTRE attacks yet continue to write in C? As rav1d shows, the perf loss is far less to migrate from C to Rust than it is to apply SPECTRE mitigations, and adding a sandbox around a memory-unsafe codec is going to be way more expensive again than using Rust code to start.

> As rav1d shows rav1d is not a full rewrite of dav1d to rust. So it really doesn't show that. It's currently C + rust + asm. I don't think we can say anything about what this does or does not prove about the performance of safe code. > Performance should not be priority #1. Security should be. Entirely depends on the application. The reason rust has `unsafe` is because there's some situations where performance needs…

Codecs are difficult and expensive to develop. Therefore they get reused in many contexts, including security critical ones. Sandboxing is shown over and over to not be a great security solution, so what this means in practice is that security-critical software that needs software decoding get pwned because software engineers don't care to prioritize it in the first place.

Why shouldn't safety be the default? If you really want to, it wouldn't be too hard to maintain a patch on top of rustc to drop the bounds checks if you want to compile object files without them.

Software decoding has a safety culture problem, and we need to talk about it.

Re: Dav2d

#130

Post by Ecco »

esposm03 wrote: post #127
thrownthatway wrote: post #84

Earlier quoted context omitted.

Is there a compelling reason encoding needs to be done locally? The point of encoding is to reduce downstream bandwidth for the viewer, and upstream bandwidth for the distribution network. The content creator only needs to upload it once.

Yes. An uncompressed 1080p, 60fps video with 24-bit color depth would need around 3Gbps to be streamed. And even if you don't need to stream it, that would still consume a sizeable portion of the write throughput of the fastest SSDs currently available; if you go up to 4K, you'd actually exceed that by a lot (not to mention, 1tb of storage would last for about 10 minutes of video).

Using raw uncompressed bitrate is a bit disingenuous. How about comparing an older, widely supported codec like H.264 as a baseline?
Post reply on HN