Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras) • Hacker News Board

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#231

Post by thingification » Tue, Jun 07, 2022, 5:06 PM UTC

ridgered4 wrote: post #188

ethanbond wrote: post #181

Earlier quoted context omitted.

Yeah I can’t believe that people think it’s possibly a desirable future for there to be no such thing as trust, and furthermore that we should accelerate our march in that direction. What an insane way to live.

It's not desirable at all, but it is inevitable and in many ways already here. While I'm not sure I agree with the argument, I think the idea is holding this stuff back just means it's more powerful and nefarious for those that do have it because the population at large will remain unaware for longer that trust is already gone. And that means innocent people are going to get hurt while that knowledge asymmetry in the…

I honestly am puzzled: the both of you (if I understand you right), do you really think some fake videos will cause there to "be no such thing as trust"? Why?

Videos aren't the only way to understand the world, and there was never any such thing as "direct observation" in the first place.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#232

Post by tablespoon » Tue, Jun 07, 2022, 5:08 PM UTC

bambax wrote: post #228

tablespoon wrote: post #227

Earlier quoted context omitted.

> The point of signing is to do a checksum of the content of the video, not just adding a signature next to the stream (which would be pretty useless). That's what I meant: sign the video content with some new key in a way that looks like it was done by a device. > The signature of the politician on a street corner would only be valid if it actually verifies the content of the video; and the only entity that can prod…

It's not about the keys of the politician. It's about the keys of the manufacturer of the camera. The aim of the signature is to prove that a video did in fact come from such camera at such location, at such time. Of course, if one has physical access to the camera/sensor it's possible to make a fake video with a valid signature. But it's a little more difficult than simply running a deepfake script on some machine i…

> It's not about the keys of the politician. It's about the keys of the manufacturer of the camera. The aim of the signature is to prove that a video did in fact come from such camera at such location, at such time.

If the goal is to allow a manufacturer to confirm video came from one of their cameras, I think it's somewhat more helpful than I was originally thinking, but it doesn't change my opinion that this technology would only "solve some authenticity problems in some narrow contexts." Namely, stuff like a burglar claiming in court that security camera video was forged to frame them. I don't think it addresses cases of embarrassing/incriminating video filmed by bystanders with rando cameras and other stuff like that.

> Of course, if one has physical access to the camera/sensor it's possible to make a fake video with a valid signature. But it's a little more difficult than simply running a deepfake script on some machine in the cloud.

If you're faking a video like I described, you certainly would have "physical access to the camera/sensor" that you claim made it. You're making a fake, which means you can concoct a story for the fake's creation involving things that are possible for you to acquire.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#233

Post by hackernewds » Tue, Jun 07, 2022, 5:09 PM UTC

barnabee wrote: post #43

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

This sounds like it's making the gun control argument regularly made in America, and I disagree with it vehemently

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#234

Post by roughly » Tue, Jun 07, 2022, 5:13 PM UTC

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

I'm reminded of Firesheep - https://en.wikipedia.org/wiki/Firesheep - which came out in 2010. It wrapped session hijacking on WiFi in an easily usable interface. The technique and the vulnerability wasn't anything new, but the extension raised awareness in a big way and really sparked a big push for getting SSL deployed, enabled, and defaulted everywhere.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#235

Post by sva_ » Tue, Jun 07, 2022, 5:14 PM UTC

oldstrangers wrote: post #224

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

Quoted post unavailable.

No post body was provided.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#236

Post by thingification » Tue, Jun 07, 2022, 5:17 PM UTC

Planktonne wrote: post #133

ramblerman wrote: post #68

Earlier quoted context omitted.

> If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. That's a dangerous precedent. Would you apply the same logic to a kitchen knife? Or if for some reason only freemium products count (not sure why), then a pentesting tool? I understand the underlying point you are trying to make but what are you proposing as an alternative exactly? Who gets to…

A knife is a multi-purpose tool that can be used as a weapon, and one that has so many important non-weapon uses that not having it would cause far more harm than having it would. This is intended primarily as a weapon, even when used defensively. There's an important qualitative difference there, though it's tempting to gallop down the slippery slope. Regardless, I'm not proposing to ban either knives or this tool -…

> we've collectively decided

I think I agree with everything you wrote except what I'm unfairly reading into this phrase: we don't need to and should not irrevocably "decide" once and for all. We might have been quite wrong to ban some poisons even in the situations that applied at the time when we banned them. If we were right to do so, times may have changed and we don't any longer. Even nuclear weapons perhaps, in certain situations.

Similar to a previous comment of mine in this thread, of course I don't know from your passing use of that word what you think about this! But I do think that that worldview, that these things are or should be "decided" in a permanent sort of way, is commonplace.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#237

Post by teakettle42 » Tue, Jun 07, 2022, 5:18 PM UTC

jacquesm wrote: post #41

teakettle42 wrote: post #37

Earlier quoted context omitted.

They sued using the argument that the manufacturer’s advertising violated Connecticut consumer law. That’s not remotely the same thing as holding a creator liable for the misuse of his work by unrelated third-parties. Frankly, the illiberal push towards “ban everything that could be misused” is disheartening, especially when we see it applied to a speech-based tool like this one.

Yes, they did that because they did what everybody in that position would do: pick the argument that is most likely to find sympathy with a jury. Also: note that the degree to which everything is labeled 'speech' is fairly uniquely American.

This argument echos the same justifications made for the US’ cryptography embargoes in the 1990s.

(Rightfully) labeling code as speech is what allowed us to break down that barrier.

Would you also be questioning the ethics of djb in 1995? https://en.wikipedia.org/wiki/Daniel_J._Bernstein#Bernstein_...

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#238

Post by kmlx » Tue, Jun 07, 2022, 5:19 PM UTC

blagie wrote: post #150

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

As someone with a stalker, I can't emphasize this enough. A stalker will go to all sorts of lengths to do bizarre shit. People don't believe it. I would guess governments will do some equivalent there-of. Democratizing access to things -- including bad things -- has a preventative effect: 1) I can guard against things I know about 2) People take me seriously if something has been democratized The worst-case scenario…

> Democratizing access to things

not to be too pedantic, but this is not "democratizing access", as that would involve print-outs/usb sticks/discs of the code distributed to people that can't access the internet, accessibility issues, bias considerations etc etc. as such, this is just "access".

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#239

Post by ethanbond » Tue, Jun 07, 2022, 5:20 PM UTC

thingification wrote: post #231

ridgered4 wrote: post #188

Earlier quoted context omitted.

It's not desirable at all, but it is inevitable and in many ways already here. While I'm not sure I agree with the argument, I think the idea is holding this stuff back just means it's more powerful and nefarious for those that do have it because the population at large will remain unaware for longer that trust is already gone. And that means innocent people are going to get hurt while that knowledge asymmetry in the…

I honestly am puzzled: the both of you (if I understand you right), do you really think some fake videos will cause there to "be no such thing as trust"? Why? Videos aren't the only way to understand the world, and there was never any such thing as "direct observation" in the first place.

Videos are considered one of the more authoritative sources of factual information. E.g. Imagine how disruptive it’d be if a deepfake of Biden announcing support for Putin’s invasion of Ukraine were to circulate during those first hours of the invasion? Sure, there are other communications channels that would be able to dispute the authenticity of that video, but would those channels be as fast and as (apparently) authoritative as video of Biden “himself” declaring his support?

Now consider a few months after this hypothetical fake-and-refutation, what if Biden actually does announce an emergency threat to America, and people end up waiting a few days (or weeks, or months) for the refutation to come.

The “no such thing as trust” is not coming from this innovation in particular but rather the confluence of two general trends: 1) heavy reliance on digital media and 2) ever-increasing ability to fake digital media.

Note that you don’t even need large, successful falsified media campaigns to create distrust. All you need is for people to know that the capability exists, and trust is immediately damaged. Even trivially-detectable falsifications can end up being difficult to sort through if there are a billion fake pieces of information crowding out a hundred real pieces of information. This is true even if all billion fakes are trivially detectable by a human (which they won’t be).

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#240

Post by thingification » Tue, Jun 07, 2022, 5:22 PM UTC

Hendrikto wrote: post #136

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

If I stab someone with a knife, who is responsible? The inventor of knives? The knife‘s manufacturer? The store who sold me the knife? I would say the responsibility lays 100% with myself. I do not think it makes sense to pursue inventors for what happens to their creations, unless they actively encourage misuse.

https://yewtu.be/watch?v=QdhwTXwhA4c

Sorry can't resist