Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

221–230 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#221

Post by bufferoverflow »

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

Do you think knife manufacturers are responsible for stabbings and knife attacks?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#222

Post by Tepix »

throw9871928 wrote: post #83
Tepix wrote: post #80

Earlier quoted context omitted.

> it will pretty much solve the trust issues I'm not so sure. How will you verify a signature if you see a video on TV or on social media? Do you believe these devices are 100% secure and the keys will never be extracted?

Nothing is 100% secure, but the point of using a TPM is to prevent extraction of the keys.

I agree that it's better than nothing. Sooner or later deepfakes will be pretty much perfect and detection will be more or less impossible.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#223

Post by avereveard »

HWR_14 wrote: post #192
blagie wrote: post #150

Earlier quoted context omitted.

As someone with a stalker, I can't emphasize this enough. A stalker will go to all sorts of lengths to do bizarre shit. People don't believe it. I would guess governments will do some equivalent there-of. Democratizing access to things -- including bad things -- has a preventative effect: 1) I can guard against things I know about 2) People take me seriously if something has been democratized The worst-case scenario…

Your "worst case" depends greatly on who the select group is. Is it movie studios making 9 figure budgets, or is it any obsessed person who can figure out how to find and install software. Obviously, it's hard to imagine many situations like that, but you can imagine a process that required a 8-figure quantum supercomputer.

history say you cannot control how selected the selected group is, no matter the implications, i.e. nuclear bomb secrets from manatthan project finding their way into russian hands

https://www.smithsonianmag.com/history/spies-who-spilled-ato...

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#224

Post by oldstrangers »

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

No post body was provided.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#225

Post by avivo »

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

Ease of access does matter.

It only buys time, but that can provide the time needed to create countermeasures and ideally make those very accessible—somewhat similar to responsible vulnerability disclosure.

This piece goes into more detail: https://aviv.medium.com/the-path-to-deepfake-harm-da4effb541... (excerpt from a working paper, part of which was presented at NeurIPs).

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#226

Post by bambax »

tablespoon wrote: post #203
jonathanstrange wrote: post #89

Earlier quoted context omitted.

It shouldn't be too hard to film a deepfake movie from a screen or projection that don't make it obvious it was filmed. That way, the cryptographic signature will even lend extra authenticity to the deepfake!

>> I hope this sort of video signing will be mainstream in all cameras in the future (i.e. cellphones etc), as it will pretty much solve the trust issues deep fakes are causing. > It shouldn't be too hard to film a deepfake movie from a screen or projection that don't make it obvious it was filmed. That way, the cryptographic signature will even lend extra authenticity to the deepfake! Would you even have to go that…

The point of signing is to do a checksum of the content of the video, not just adding a signature next to the stream (which would be pretty useless).

The signature of the politician on a street corner would only be valid if it actually verifies the content of the video; and the only entity that can produce that signature is the holder of the private key.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#227

Post by tablespoon »

bambax wrote: post #226
tablespoon wrote: post #203

Earlier quoted context omitted.

>> I hope this sort of video signing will be mainstream in all cameras in the future (i.e. cellphones etc), as it will pretty much solve the trust issues deep fakes are causing. > It shouldn't be too hard to film a deepfake movie from a screen or projection that don't make it obvious it was filmed. That way, the cryptographic signature will even lend extra authenticity to the deepfake! Would you even have to go that…

The point of signing is to do a checksum of the content of the video, not just adding a signature next to the stream (which would be pretty useless). The signature of the politician on a street corner would only be valid if it actually verifies the content of the video; and the only entity that can produce that signature is the holder of the private key.

> The point of signing is to do a checksum of the content of the video, not just adding a signature next to the stream (which would be pretty useless).

That's what I meant: sign the video content with some new key in a way that looks like it was done by a device.

> The signature of the politician on a street corner would only be valid if it actually verifies the content of the video; and the only entity that can produce that signature is the holder of the private key.

The hypothetical politician's encryption keys are irrelevant. The point is that even authentic video is going to be signed by some rando device with no connection to the people depicted, so a deepfake signed by some rando keys looks the same.

IMHO, cameras that embed a signature in the video stream solves some authenticity problems in some narrow contexts, but it's nowhere near a panacea that "will pretty much solve the trust issues deep fakes are causing."

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#228

Post by bambax »

tablespoon wrote: post #227
bambax wrote: post #226

Earlier quoted context omitted.

The point of signing is to do a checksum of the content of the video, not just adding a signature next to the stream (which would be pretty useless). The signature of the politician on a street corner would only be valid if it actually verifies the content of the video; and the only entity that can produce that signature is the holder of the private key.

> The point of signing is to do a checksum of the content of the video, not just adding a signature next to the stream (which would be pretty useless). That's what I meant: sign the video content with some new key in a way that looks like it was done by a device. > The signature of the politician on a street corner would only be valid if it actually verifies the content of the video; and the only entity that can prod…

It's not about the keys of the politician. It's about the keys of the manufacturer of the camera. The aim of the signature is to prove that a video did in fact come from such camera at such location, at such time.

Of course, if one has physical access to the camera/sensor it's possible to make a fake video with a valid signature. But it's a little more difficult than simply running a deepfake script on some machine in the cloud.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#229

Post by thingification »

teakettle42 wrote: post #208
light_hue_1 wrote: post #200

Earlier quoted context omitted.

That's like saying that nuclear weapons exist, and bad actors can potentially get them, so let's lower the bar so that anyone can. Making such tools accessible is reprehensible. It will lead to more bad actors, to less trust in media and in any objective reality, and more erosion of our institutions and society. There is absolutely no reason whatsoever for this. It's unethical and frankly downright evil.

On par with nuclear weapons? Downright evil? You’re being absurd. The technology exists, and it’ll get better. Pretending that it doesn’t exist or banning it won’t make it go away — it’ll just be used by the least scrupulous and most powerful. Disruptive efforts like this are most upsetting to anxiety-ridden people who think that if they could just control things firmly enough, everyone and everything will be safe. T…

I think you describe the conventional wisdom well here.

But: are those really the only options? Paranoid stop-the-world wishful thinking or all-new-trends-are-inevitable?

I realise I'm almost certainly wildly misrepresenting your point of view with the latter label! I don't really mean to ascribe it to you. But I think the conversation often ends up tacitly as a debate between those two positions, both of which are mistakes. But one other option -- as an example -- is to find more complicated ways to stop doing .

A striking historical example to me because it's so old it can hardly be said to have been an invention that needed to be responded to -- rather just an apparent invitable fact of life: For the longest time, it seems everyday brutal violence on a level we find hard to imagine was common. At the time, that would have been obviously inevitable. Nothing else had ever happened. Then it basically stopped -- turns out it never was inevitable. Why? We didn't for example ban knives as a technology, but we did a lot of things over the centuries that, to a good numerical approximation, stopped us using them for violence (banning them in certain situations is only one, I suspect not the most important and maybe not even necessary).

Post reply on HN