Live data from Hacker News

Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

news.ycombinator.com

31–40 of 93 posts

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#31

Are you concerned that your product will inadvertently improve deepfakes? Suddenly you've given them a baseline that they need to be better than, and hackers love challenges. I predict this will turn into a constant arms race like AV or copyright protection, and I don't think this will work in the long run. IMO, KYC needs to go back to in person verification. Everything you can do digitally can be faked or impersonat…

I came here to write that. Me and my friends have a competition to get a (obviously CGI) picture of my living room filled with gold coins past those "is that photo real" platforms. So far, none has stood up to the test.

The worst one so far was TruePic, who even gave me a certificate of authenticity and then kept that certification PDF online on their website until they heard about us openly mocking them, and then they blocked the photo not because someone noticed that it's fake, but because "User has violated terms of service", which was me mocking them.

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#32

Are you concerned that your product will inadvertently improve deepfakes? Suddenly you've given them a baseline that they need to be better than, and hackers love challenges. I predict this will turn into a constant arms race like AV or copyright protection, and I don't think this will work in the long run. IMO, KYC needs to go back to in person verification. Everything you can do digitally can be faked or impersonat…

If I was them I'd be concerned that it DOESNT move in this direction. Not a lot of money to be made selling one-and-done fake detection software, but a fortune available if they can convince clients a perpetual subscription is a core requirement.

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#33
post #21

Imho this is a good product, but wouldn’t it make more sense to simply sign videos cryptographically? Unfakeable and unbeatable. So someone uploads a video, you sign it, they display video. If authenticity is in question check the signature. Deep fake detection is intractable imo. Use cryptography instead. Hell if you want to be thorough sign each frame and create an extension for YouTube and other providers to liter…

Most videos people watch nowadays don't come from "official" authorities, whose public keys are known and signatures can be verified. When videos are recorded by random people from their smartphones, what signature are you going to verify? Even if smartphone manufacturers start integrating digital signatures right into their cameras, you can use the smartphone to re-record a pre-recorded fake video. And I'm sure that…

" you can use the smartphone to re-record a pre-recorded fake video."

I assume the intent is not to store solely a direct hash of the original video, but also the original file which can be fingerprinted, or the fingerprint itself that will be matched if a later duplicate is uploaded.

Fingerprint differing from hashing in that two non-identical, but similar files (e.g. an original video and deepfake) can have the same fingerprint, but differing checksums.

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#35
post #25

Earlier quoted context omitted.

So the folks at the forefront of deep fake technology (i.e. the attackers you're targeting) will slip through your product because it lags behind the state of the art (like AV, which you said is the approach you're following), while innocent folks will be caught by it due to a new kafkaesque version of "prove you're not a bot" since you focus on reducing false negatives. Hopefully I can avoid companies using your pro…

Retrospective antivirus-esque techniques are still useful, though, as not every actor is a state-level actor, and even then, forcing state-level actors to "burn" their state-of-the-art exploits/models because previous exploits/models are detected out-of-the-box, slows down the abuse of those actors. And realistically, since deepfake detection will inevitably be more expensive than captchas or antivirus scanning, this…

We agree. Dataset fidelity and bias are major concerns for publicly available datasets. For this reason we are working to develop programmatically created datasets along with anti-bias testing and policies.

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#36
I too wonder whether cryptographic signatures will be the long run solution to deepfakes. Can you outline why you don't think that will be the case?

Here's an alt solution to argue against:

1. The necessary PKI gets bootstrapped by social media companies, where deepfakes begin to seriously threaten their "all-in-on-video" strategy, and simultaneously look like an opportunity to layer on some extra blue-check-style verification.

2. Example: you upload your first video to twitter, it sees the AV streams are unsigned, generates a keypair for you, does the signing, and adds the pubkey half to your twitter account. (All of this could be done with no user input.)

3. The AV streams have signatures on short sequences of frames. Say every 24th video frame has a signature over the previous 24 frames embedded into the picture. Similarly, every second of audio has a signature baked in.

4. The signatures aren't metadata that can be easily thrown away. They're watermarked into the picture and audio themselves in a way that's ~invisible & ~inaudible, but also robust to downsampling & compression. This is already technically possible.

5. Since we're signing every past second of the AV streams, this works for live video.

6. Viewers on the platform see a green check on videos with valid signatures; maybe they even see the creator's twitter handle if this is a reshare.

7. Like all social media innovations, the other major platforms copy it within the next 6 - 12 months. POTUS uses it. People come to expect it.

8. Long run: the public comes to regard any unverified video footage with suspicion.

Why won't this deepfake solution come to pass in the long run?

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#37
First of all, congratulations on the lauch! Your description about the "model of models" and combining their scores is really intriguing. Detecting deepfake is an interesting topic on its own and apparently there are lots of use cases that I'm not even aware of, partly due to my limited knowledge in this subject. There are a few points of I'm curious about (Beware that the following questions can be very silly, coming from someone having little to no experience in the field):

- What do you use as input for the model? Does it use all the pixels in all the frames in the input video? How about the video's metadata (location, extension,...)?

- My biggest concern about fighting deepfakes is that they have a point to achieve where the line between reality and fiction is nonexistent. Namely, if a deepfake video of someone can be created to look exactly like a real one if that someone decide to record such a video, I imagine there would be no way to tell the deepfake video from the authentic one (since there is no difference between the two). Because of that, this looks like a losing battle to me, but maybe I'm just too pessimistic. Do you feel that it is a real problem? Do you believe it is such a long shot that we shouldn't be worried about, or even if things reach that point, there would still be tools in our arsenal to counter such technologies.

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#38
post #36

I too wonder whether cryptographic signatures will be the long run solution to deepfakes. Can you outline why you don't think that will be the case? Here's an alt solution to argue against: 1. The necessary PKI gets bootstrapped by social media companies, where deepfakes begin to seriously threaten their "all-in-on-video" strategy, and simultaneously look like an opportunity to layer on some extra blue-check-style ve…

(Not OP)

This would work for videos that people want to acknowledge as their own, but they could just tweet out they own it for the same affect.

The issue this tackles is videos that people don't want to claim ownership of, e.g. if a video emerged of kicking a child, the politician can't say "I haven't signed it therefore it's not mine", instead we need tools like the above to be able to say, "this is faked, do not trust it".

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#39
post #36

I too wonder whether cryptographic signatures will be the long run solution to deepfakes. Can you outline why you don't think that will be the case? Here's an alt solution to argue against: 1. The necessary PKI gets bootstrapped by social media companies, where deepfakes begin to seriously threaten their "all-in-on-video" strategy, and simultaneously look like an opportunity to layer on some extra blue-check-style ve…

(Not OP) This would work for videos that people want to acknowledge as their own, but they could just tweet out they own it for the same affect. The issue this tackles is videos that people don't want to claim ownership of, e.g. if a video emerged of kicking a child, the politician can't say "I haven't signed it therefore it's not mine", instead we need tools like the above to be able to say, "this is faked, do not t…

Maybe in the interim we need deepfake detection tools, but I'm asking about the long run. Suppose signed AV takes off as described above. The public has been trained: "If the video doesn't have The King's Seal, it's not from The King."

Re: Launch HN: Reality Defender (YC W22) – Deepfake Detection Platform

#40
post #35
post #25

Earlier quoted context omitted.

Retrospective antivirus-esque techniques are still useful, though, as not every actor is a state-level actor, and even then, forcing state-level actors to "burn" their state-of-the-art exploits/models because previous exploits/models are detected out-of-the-box, slows down the abuse of those actors. And realistically, since deepfake detection will inevitably be more expensive than captchas or antivirus scanning, this…

We agree. Dataset fidelity and bias are major concerns for publicly available datasets. For this reason we are working to develop programmatically created datasets along with anti-bias testing and policies.

"Bias" and "anti-bias" is a slippery snake that will bite you as soon as it warms up to you.
Post reply on HN