Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

61–70 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#61
post #60

Earlier quoted context omitted.

It's not what we think about this, it is what the creators think about it that matters.

It doesn't matter what anyone thinks about it.

I actually disagree with that. I think software and ethics are closely intertwined and too often we see them as disjoint. It's interesting to try to find a software project that does not have an ethically dubious angle to it, that isn't all that simple.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#62

Governments and bank branches should offer physical identification as a (potentially payed) service globally, and provide a digital signature that a person is owning a physical device (which should be valid until the device is reset or invalidated by the person).

I always imagined the USPS fulfilling this need.

I don't know how serious is USPS, but in my country the post office doesn't check if the signature looks the same for example.

Banks face and handle fraud on a much higher level, and have a much better training in identificacion.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#63
post #4

Hi there, one of the maintainers here. Thanks for sharing & AMA!

Even if you claim that you are legally not responsible for any abuses of this software do you feel any moral responsibility?

Isn't the situation completely analogous to other "ethical hacking" software?

I distinctly remember a very similar discussion around firesheep 12 years ago: https://www.computerworld.com/article/2469667/firesheep-fire...

https://news.ycombinator.com/item?id=1828955

> There are probably going to be a lot of people negatively affected by this for quite some time to come. One thing to point out is that there are grades of things. There is "public", and then there is "top hit on Google". Similarly, there is "insecure" and then there is "simple doubleclick tool to facilitate identity theft".

> How many millions of dollars and man hours is it going to take to lock down every access point? How many new servers are going to be needed now that https is used for everything and requests can't be cached?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#64

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

Does this apply to nuclear weapons? To biological weapons?

“Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#65

Earlier quoted context omitted.

But if such things exist in secret and there's neither awareness nur a reference implementation to develop countermeasures against, wouldn't that also cause harm?

Absolutely, and there's definitely a strong argument to be made for increasing awareness of existing tools and attack vectors. Importantly though, that aim is served by proofs-of-concept and information sharing more than it is by releasing point-and-shoot attacks. This comment puts it rather well [1]. As above, I'm not saying this shouldn't have been created, or even that it shouldn't have been released. I am saying…

OpenAI attempted this with GPT-2’s incremental release strategy and was lambasted on moral and ethical grounds as well.

https://openai.com/blog/better-language-models/

So individuals working on a technology like this have no clear societal consensus to use as a guide when making the decision. Strong arguments can be made in both directions.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#66
post #16

Earlier quoted context omitted.

It's the same as the age old "guns don't kill people" that seems to work fine for the weapons industry in the US.

The difference is that guns are literally deigned for killing people. There are nonlethal alternatives, so when people buy guns (the exception being hunting) the purpose it to be able to kill someone.

There are non-lethal alternatives to devices "literally deigned [sic] for killing people?"

This logically does not compute.

I guess you mean in situations where guns are currently used, other strategies could be employed?

Since the goal of a firearm is to hurl a chunk of metal through the air at high speed, I can't really think of alternatives. Besides air rifles, we just have traditional powder guns filling that niche.

The usefulness of firearms extends beyond killing people, and I don't think "guns are literally designed for killing people" is very accurate. I think the majority of guns are not designed for killing people. I think the Ruger 10/22 is the best-selling gun of all time, and it's .22 caliber, which is definitely not what you'd choose if the goal of your design was to kill people.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#67
post #63

Earlier quoted context omitted.

Even if you claim that you are legally not responsible for any abuses of this software do you feel any moral responsibility?

Isn't the situation completely analogous to other "ethical hacking" software? I distinctly remember a very similar discussion around firesheep 12 years ago: https://www.computerworld.com/article/2469667/firesheep-fire... https://news.ycombinator.com/item?id=1828955 > There are probably going to be a lot of people negatively affected by this for quite some time to come. One thing to point out is that there are grades…

Indeed it is, but I'm interested in this particular author's stance on this, prompted by their disclaimer which clearly indicates that they realize that there is the risk of abuse, and an AMA seems to be an excellent opportunity to gain some insight.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#68

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

> If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse.

That's a dangerous precedent. Would you apply the same logic to a kitchen knife? Or if for some reason only freemium products count (not sure why), then a pentesting tool?

I understand the underlying point you are trying to make but what are you proposing as an alternative exactly? Who gets to decide which products fall into a gray zone, whilst others are only for bad use. We already see this kind of shoddy thinking leading to keeping DALE-2 out of the public's hands (or at least that is their claim).

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#70

Earlier quoted context omitted.

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.

No, but it does apply to the distribution of slippery slopes.

The OP specifically said tools "like this"

Post reply on HN