Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

41–50 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#41

Post by jacquesm »

teakettle42 wrote: post #37
jacquesm wrote: post #21

Earlier quoted context omitted.

The world is larger than the US, and even there that argument does not hold water: https://www.nytimes.com/2022/02/15/nyregion/sandy-hook-famil...

They sued using the argument that the manufacturer’s advertising violated Connecticut consumer law. That’s not remotely the same thing as holding a creator liable for the misuse of his work by unrelated third-parties. Frankly, the illiberal push towards “ban everything that could be misused” is disheartening, especially when we see it applied to a speech-based tool like this one.

Yes, they did that because they did what everybody in that position would do: pick the argument that is most likely to find sympathy with a jury.

Also: note that the degree to which everything is labeled 'speech' is fairly uniquely American.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#42

Post by tgsovlerkhgsel »

krageon wrote: post #24
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

This has a clear benign use. Of course it sucks that you can also use it in a hostile manner - but the fact that this tool is publicly available rather than hidden in the pocket of some unscrupulous blackhat means that every space that uses verification with these methods can now incorporate this type of testing. That's a net benefit for society. I do think disclaimers like this are a little juvenile (it reeks of a U…

More generically, I think there's a big difference between releasing proof of concepts, and fully weaponized tools. While the latter is also usable by red teams, it also gives attackers (who often wouldn't have the resources to build the same) the weapons they need.

Personally, I like when people release proof of concepts, and hate it when they release weaponized tools. Especially when I inevitably end up reading reports where APT groups are using those tools.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#43

Post by barnabee »

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries.

IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#44

Post by Planktonne »

carnitine wrote: post #8
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

That’s a legal disclaimer, not a moral one.

I think that's part of what makes me uneasy about this - it's a solely legal disclaimer for something with a moral dimension, and it reads to me like the abdication of responsibility, rather than just a shield against litigation.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#45

Post by dividedbyzero »

Planktonne wrote: post #40
wdroz wrote: post #13

Earlier quoted context omitted.

> If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I don't agree, but people will never reach consensus on this moral topic. So please don't call the other side "at best naive".

If there's no duty to consider potential harm when releasing tools, then why would there be a duty to avoid criticising people who release them?

But if such things exist in secret and there's neither awareness nur a reference implementation to develop countermeasures against, wouldn't that also cause harm?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#46

Post by vsnf »

jacquesm wrote: post #10
giorgiop wrote: post #4

Hi there, one of the maintainers here. Thanks for sharing & AMA!

Even if you claim that you are legally not responsible for any abuses of this software do you feel any moral responsibility?

What kind of answer to your question would you accept? What kind of answer would you be very satisfied with, versus merely be placated by? I'm curious because I have my own answer to your question for my own things that I suspect you would be unhappy with, and I'm wondering how absolute-good you require your interrogatees to be.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#47

Post by dividedbyzero »

tgv wrote: post #30
jl6 wrote: post #28

Which requires more computation: a real-time deepfake or real-time deepfake detection? That will determine the future balance of power.

You don't need to detect a deepfake in the same time as it takes to generate one, do you?

But you may have to go through orders of magnitude more material

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#48

Post by Planktonne »

barnabee wrote: post #43
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

> there is a moral imperative to create and release tools like this as free and open source software so that _anyone_ has access to them

That's an argument I have a lot of time for, but it needs to be made, rather than - as at present - having the whole issue sidestepped. With any tool like this, I think there's a need for ethical due diligence as much as there is for engineering, and there's no evidence that this aspect has been considered at all.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#49

Post by Planktonne »

tehbeard wrote: post #35
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

So, if we take your naive take to its logical conclusion; The Apache Foundation should be considered partly responsible for all the malware distributed via their http software?

There is a conceptual difference between releasing a weapon (even if for research/defensive purposes) and releasing tools which could later be adapted negatively. That's not to say that weapons should never be created/released, but that there is an extra onus on the creators to at least consider the harm they are enabling and possible mitigations.

The Apache Foundation - great example, thank you - despite not creating weapons, has clearly put thought into how their work interacts with wider society and how to ensure positive outcomes as far as possible [1]. People absolutely don't have to agree on moral issues, but it is irresponsible not to have considered them.

[1] https://apache.org/theapacheway/index.html

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#50

Post by mjbeswick »

t0mas88 wrote: post #16
jacquesm wrote: post #9

Earlier quoted context omitted.

It is an attempt at a legal disclaimer. It may not hold up in court.

It's the same as the age old "guns don't kill people" that seems to work fine for the weapons industry in the US.

The difference is that guns are literally deigned for killing people. There are nonlethal alternatives, so when people buy guns (the exception being hunting) the purpose it to be able to kill someone.
Post reply on HN