Live data from Hacker News

Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

realitydefender.com

21–30 of 70 posts

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#21
post #4

Yeah but does it actually work, though? There have been a lot of online tools claiming to be "AI detectors" and they all seem pretty unreliable. Can you talk us through what you look for, the most common failure modes and (at suitably high level) how you dealt with those?

We've actually deployed to several Tier 1 banks and large enterprises already for various use-cases (verification, fraud detection, threat intelligence, etc.). The feedback that we've gotten so far is that our technology is high accuracy and a useful signal.

In terms of how our technology works, our research team has trained multiple detection models to look for specific visual and audio artifacts that the major generative models leave behind. These artifacts aren't perceptible to the human eye / ear, but they are actually very detectable to computer vision and audio models.

Each of these expert models gets combined into an ensemble system that weighs all the individual model outputs to reach a final conclusion.

We've got a rigorous process of collecting data from new generators, benchmarking them, and retraining our models when necessary. Often retrains aren't needed though, since our accuracy seems to transfer well across a given deepfake technique. So even if new diffusion or autoregressive models come out, for example, the artifacts tend to be similar and are still caught by our models.

I will say that our models are most heavily benchmarked on convincing audio/video/image impersonations of humans. While we can return results for items outside that scope, we've tended to focus training and benchmarking on human impersonations since that's typically the most dangerous risk for businesses.

So that's a caveat to keep in mind if you decide to try out our Developer Free Plan.

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#22

How do you prevent bad actors from using your tools as a feedback loop to tune models that can evade detection?

You would need thousands to tens of thousands of images, not just 50 to produce an adversarial network that could use the API as a check.

If someone wanted to buy it, I'm sure reality defender has protection especially because you can predict adversarial guesses.

It would be trivial for them to build "this user is sending progressively more realistic, rapid responses" if they haven't built that already.

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#23

How do you prevent bad actors from using your tools as a feedback loop to tune models that can evade detection?

We see who signs up for Reality Defender and instantly notice traffic patterns and other abnormalities that allow us to see if an account is in violation of terms of service. Also, our free tier is capped at 50 free scans a month which will not allow for said attackers to discern any tangible learnings or tactics they can use to bypass our detection models.

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#26

I feel like this will be the next big cat and mouse sega after ad-blockers; 1) Produce AI tool 2) Tool gets used for bad 3) Use anti-AI/AI detection to avoid/check for AI tool 4) AI tool introduces anti-anti-AI/detection tools 5) Repeat

This is definitely a concern, but this is more or less how the cybersecurity space already works. Having dedicated researchers and a good business model helps a lot for keeping detectors like RD on the forefront of capabilities.

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#27

I feel like a much easier solution is enforcing data provinence. Ssl for media hash, attach to metadata. The problem with AI isnt the fact its ai, its that people can invest little effort to sway things with undue leverage. A single person can look like 100's with signficantly less effort than previously. The problem with ai content is it makes abuse of public spaces much easier. Forcing people to take credit for wor…

I understand the appeal of hashing-based provenance techniques, though they’ve faced some significant challenges in practice that render them ineffective at best. While many model developers have explored these approaches with good intentions, we’ve seen that they can be easily circumvented or manipulated, particularly by sophisticated bad actors who may not follow voluntary standards.

We recognize that no detection solution is 100% accurate. There will be occasional false positives and negatives. That said, our independently verified an internal testing shows we’ve achieved the lowest error rates currently available for addressing deepfake detection.

I’d respectfully suggest that dismissing AI detection entirely might be premature, especially without hands-on evaluation. If you’re interested, I’d be happy to arrange a test environment where you could evaluate our solution’s performance firsthand and see how it might fit your specific use case.

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#28
How easy is it to fool Reality Defender into making false positives?

Whenever I'm openly performing nefarious illegal acts in public, I always wear my Sixfinger, so if anyone takes a photo of me, I can plausibly deny it by pointing out (while not wearing it) that the photo shows six fingers, and obviously must have been AI generated.

In support of said nefarious illegal acts, the Sixfinger includes a cap-loaded grenade launcher, gun, fragmentation bomb, ballpoint pen, code signaler, and message missile launcher. It's like a Swiss Army Finger! You can 3d print a cool roach clip attachment too.

"How did I ever get along with five???"

https://www.youtube.com/watch?v=ElVzs0lEULs

https://www.museumofplay.org/blog/sixfinger-sixfinger-man-al...

https://www.museumofplay.org/app/uploads/2010/11/Sixfinger-p...

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#29

It's sadly not often enough I see a young company doing work that I feel only benefits society, but this is one of those times, so thank you and congratulations.

Thank you! We’ve been working on this since 2021 (and some of us a bit before that), and we’re reminded every day that we are ultimately working something that helps people on the macro and micro level. We want a world free of the malevolent uses of deepfakes for ourselves, our loved ones, and everyone beyond, and feel all should be privy to such protection.

Re: Launch HN: Reality Defender (YC W22) – API for Deepfake and GenAI Detection

#30

First want to say that I sincerely appreciate you working on this problem. The proliferation of deepfakes is something that virtually every technology industry is dealing with right now. Suppose that deepfake technology progressed to the point where it is still detectable by your technology, but is impossible for the naked eye. In that scenario (which many would call an eventuality), wouldn't you also be compelled to…

Thank you. As an inference-based detection platform, our models go into every scan with the assumption that all files are both not the original/ground truth AND the files have been likely transcoded. We never say something is 0% or 100% fake because we don’t have that ground truth. That said, our award-winning models are able to say, with a confidence score of 1-99% — the higher being likely manipulated — which, in turn, is sent to the team using said detection to action as they will. Some use it as one of many signals to make an informed decision manually. Others have chosen to moderate or label accordingly. There are experts who’ve been called to testify on matters like this one, and some of them work on these very models.

As for synthetic content that is undetectable to the naked eye or ear, we are already there.

Post reply on HN