Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dav2d

code.videolan.org

Post reply on HN
Print view View original

91–100 of 199 posts

Re: Dav2d

#91

Post by nerdralph »

thresh wrote: post #61
nerdralph wrote: post #54

Earlier quoted context omitted.

I highly doubt there is no other technically feasible option to block the AI bots. You end up blocking not just bots, but many humans too. When I clicked on the link and the bot block came up, I just clicked back. I think HN posts should have warnings when the site blocks you from seeing it until you somehow, maybe, prove you are human.

I'm all ears on how we can fix it otherwise. Keep in mind that those kinds of services: - should not be MITMed by CDNs - are generally ran by volunteers with zero budget, money and time-wise

First off, don't block the first connection of the day from a given IP. Rate limit/block from there, for example how sshguard does it.

I've seen several posts on HN and elsewhere showing many bots can be fingerprinted and blocked based on HTTP headers and TLS.

For the bots that perfectly match the fingerprint of an interactive browser and don't trigger rate limits, use hidden links to tarpits and zip bombs. Many of these have been discussed on HN. Here's the first one that came to memory: https://news.ycombinator.com/item?id=42725147

Re: Dav2d

#92

Post by fragmede »

thresh wrote: post #81
hectormalot wrote: post #65

Earlier quoted context omitted.

Maybe I’m naive about this, but I didn’t expect AI scrapers to be that big of a load? I mean, it’s not that they need to scrape the same at 1000+ QPS, and even then I wouldn’t expect them to download all media and images either? What am I missing that explains the gap between this and “constant DDoS” of the site?

You cant really cache the dynamic content produced by the forges like Gitlab and, say, web forums like phpbb. So it means every request gets through the slow path. Media/JS is of course cached on the edge, so it's not an issue. Even when the amount of AI requests isnt that high - generally it's in hundreds per second tops for our services combined - that's still a load that causes issues for legitimate users/develope…

You kind of can though. You serve cached assets and then use JavaScript to modify it for the individual user. The specific user actions can't be cached, but the rest of it can.

Re: Dav2d

#93

Post by BLKNSLVR »

delfinom wrote: post #25
jzebedee wrote: post #2

Project description: dav2d is the fastest AV2 decoder on all platforms :) Targeted to be small, portable and very fast. If you're out of the loop like me: AV2 is the next-generation video coding specification from the Alliance for Open Media (AOMedia). Building on the foundation of AV1, AV2 is engineered to provide superior compression efficiency, enabling high-quality video delivery at significantly lower bitrates.…

looks at if AV2 is dead in the water https://www.sisvel.com/insights/av2-is-coming-sisvel-is-prep... yep

You can tell Sisvel are a bunch of grifters by the fact they use slight grey text on a slightly less grey background.

Aesthetics over function; style over substance. If that's their web design policy it's likely their policy in all other aspects.

I'm also not sure that they're aware that intellectual property rights no longer exist in the US. If AV2 was vibe coded, there would be no case.

Re: Dav2d

#94

Post by adobrawy »

IshKebab wrote: post #55
Almondsetat wrote: post #44

Earlier quoted context omitted.

What are you even implying?

That codecs should be written in safer languages given that they usually process untrusted files. There have been a number of serious hacks from file parsing bugs due to them being written in unsafe languages. There's literally a DSL designed for this purpose (Wuffs) so it would be interesting to hear why they didn't use it.

There was already attempt for dav1d to re-implement in rust by rav1d. It was hard to match performance: https://www.memorysafety.org/blog/rav1d-perf-bounty/

Re: Dav2d

#95

Post by astrange »

IshKebab wrote: post #55
Almondsetat wrote: post #44

Earlier quoted context omitted.

What are you even implying?

That codecs should be written in safer languages given that they usually process untrusted files. There have been a number of serious hacks from file parsing bugs due to them being written in unsafe languages. There's literally a DSL designed for this purpose (Wuffs) so it would be interesting to hear why they didn't use it.

The people who write DSLs for video codec asm, or who claim that it's fine to use intrinsics or X higher-level language and it will still be fast enough to be usable, are simply wrong and have never been able to demonstrate otherwise.

Having said that I do think you could write a DSL to generate safe performant asm for a video codec. Just not a platform-independent one. It would still have to be asm.

Re: Dav2d

#96

Post by astrange »

fguerraz wrote: post #28
dcsommer wrote: post #18

We must not continue to develop media codecs in memory unsafe languages. Small, auditable sections can opt-out perhaps, but choosing default-unsafe for this type of software is close to professional negligence.

Cryptography and video codecs are notable exceptions, they put a lot of effort to making the code provably memory safe: no recursion, limited use of stack variables, no dynamic allocations, etc. As a result, memory safe languages bring nothing but trouble by making it non deterministic, that’s especially true for crypto where compiler “optimisations” guarantee you side channels attacks.

Video codecs just don't need to do dynamic allocations because it's not relevant to the problem. There's still certainly plenty of opportunities for memory bugs because there's a lot of pointer math.

Re: Dav2d

#97

Post by astrange »

BLKNSLVR wrote: post #93
delfinom wrote: post #25

Earlier quoted context omitted.

looks at if AV2 is dead in the water https://www.sisvel.com/insights/av2-is-coming-sisvel-is-prep... yep

You can tell Sisvel are a bunch of grifters by the fact they use slight grey text on a slightly less grey background. Aesthetics over function; style over substance. If that's their web design policy it's likely their policy in all other aspects. I'm also not sure that they're aware that intellectual property rights no longer exist in the US. If AV2 was vibe coded, there would be no case.

> If AV2 was vibe coded, there would be no case.

…for copyright. Not for anything else. Patents would still apply.

Re: Dav2d

#98

Post by simonask »

fguerraz wrote: post #28
dcsommer wrote: post #18

We must not continue to develop media codecs in memory unsafe languages. Small, auditable sections can opt-out perhaps, but choosing default-unsafe for this type of software is close to professional negligence.

Cryptography and video codecs are notable exceptions, they put a lot of effort to making the code provably memory safe: no recursion, limited use of stack variables, no dynamic allocations, etc. As a result, memory safe languages bring nothing but trouble by making it non deterministic, that’s especially true for crypto where compiler “optimisations” guarantee you side channels attacks.

What in the world do you mean by “non-deterministic”?

C compilers, Rust compilers, and assemblers are all deterministic.

Re: Dav2d

#99

Post by simonask »

fishgoesblub wrote: post #37
dcsommer wrote: post #18

We must not continue to develop media codecs in memory unsafe languages. Small, auditable sections can opt-out perhaps, but choosing default-unsafe for this type of software is close to professional negligence.

Of the 3 software AV1 encoders, the only one that is fully dead is the Rust encoder (rav1e). If people truly wanted memory safe encoders/decoders, they would fund and develop them.

Encoding is a way, way less risky thing to be doing compared to decoding.

Re: Dav2d

#100

Post by breve »

kylec wrote: post #47

I wonder if the author is a Dave2D fan? https://www.youtube.com/@Dave2D

The AV1 decoder is dav1d. The AV2 decoder is dav2d.

One day in the mysterious future the AV3 decoder will be dav3d.

Post reply on HN