Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

81–90 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#81

Post by ethanbond »

rlayton2 wrote: post #70
ethanbond wrote: post #64

Earlier quoted context omitted.

Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.

No, but it does apply to the distribution of slippery slopes. The OP specifically said tools "like this"

I guess I don’t understand what “like this” means.

While it’s convenient to dismiss any discussion about A and something slightly similar to A as “slippery slope,” an inconvenient fact is that the slope often is slippery. It’s increasingly evident to anyone paying attention that information warfare can yield disastrous effects in a variety of ways: disrupting peaceful transition of power; disrupting critical infrastructure; disrupting command and control of ”traditional” WMD systems, even.

So that’s sort of the spirit of my question: how far does this argument go? What is the definition of “like this?”

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#82

Post by Francesco_deep »

ramblerman wrote: post #68
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

> If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. That's a dangerous precedent. Would you apply the same logic to a kitchen knife? Or if for some reason only freemium products count (not sure why), then a pentesting tool? I understand the underlying point you are trying to make but what are you proposing as an alternative exactly? Who gets to…

I sell airplanes do I am to feel responsible for the 9/11?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#83

Post by throw9871928 »

Tepix wrote: post #80
throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

> it will pretty much solve the trust issues I'm not so sure. How will you verify a signature if you see a video on TV or on social media? Do you believe these devices are 100% secure and the keys will never be extracted?

Nothing is 100% secure, but the point of using a TPM is to prevent extraction of the keys.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#84

Post by IfOnlyYouKnew »

barnabee wrote: post #43
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

I can’t see a way this technology could be used defensively. Wide access just leads to more abuse. There’s no principle by which it serves some sort of justice to make some crime more accessible.

It’s surprising how often this argument is made compared to, say, equal-opportunity access to fancy cars or good housing.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#85

Post by coldcode »

throw457 wrote: post #54
jl6 wrote: post #28

Which requires more computation: a real-time deepfake or real-time deepfake detection? That will determine the future balance of power.

Deepfakes are for deceiving humans. There are still way too many artifacts in the outputs that are easy to detect in code.

The problem comes that even with extremely good deepfake detection, political operations can still use the deepfakes to fool the average voter into believing that is it real, irrespective of any successful analysis. You and I might not be fooled but we don't matter if 10000X people are fooled and affect our lives anyway in a negative way.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#86

Post by IfOnlyYouKnew »

tehbeard wrote: post #35
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

So, if we take your naive take to its logical conclusion; The Apache Foundation should be considered partly responsible for all the malware distributed via their http software?

That’s not a naive take but a stupid one, and it wasn’t made by OP. Slippery slope is a fallacy, not an argument.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#87

Post by [deleted] »

yosito wrote: post #57

A few months ago, the IRS made me verify my identity using some janky video conferencing software where I had to hold up a copy of my passport. The software was so hard to use, that I can't believe average people manage to do it. Now, real-time deep fakes are literally easier to create than using the video verification software itself. This will have interesting societal implications.

[deleted]

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#88

Post by ls15 »

tgsovlerkhgsel wrote: post #42
krageon wrote: post #24

Earlier quoted context omitted.

This has a clear benign use. Of course it sucks that you can also use it in a hostile manner - but the fact that this tool is publicly available rather than hidden in the pocket of some unscrupulous blackhat means that every space that uses verification with these methods can now incorporate this type of testing. That's a net benefit for society. I do think disclaimers like this are a little juvenile (it reeks of a U…

More generically, I think there's a big difference between releasing proof of concepts, and fully weaponized tools. While the latter is also usable by red teams, it also gives attackers (who often wouldn't have the resources to build the same) the weapons they need. Personally, I like when people release proof of concepts, and hate it when they release weaponized tools. Especially when I inevitably end up reading rep…

Is a pen that contains ink a weaponized tool?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#89

Post by jonathanstrange »

throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

It shouldn't be too hard to film a deepfake movie from a screen or projection that don't make it obvious it was filmed. That way, the cryptographic signature will even lend extra authenticity to the deepfake!

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#90

Post by Reubensson »

Planktonne wrote: post #48
barnabee wrote: post #43

Earlier quoted context omitted.

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

> there is a moral imperative to create and release tools like this as free and open source software so that _anyone_ has access to them That's an argument I have a lot of time for, but it needs to be made, rather than - as at present - having the whole issue sidestepped. With any tool like this, I think there's a need for ethical due diligence as much as there is for engineering, and there's no evidence that this as…

To me it sounds like it boils down to either you release the software or you don't. If you decide to release it into public, I don't think there is much you can realistically do to prevent abuse.
Post reply on HN