Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

281–290 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#281

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

[deleted]

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#282

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

Same applies to a lot of security oriented tools, most notably metasploit which had these same accusations when it came out long ago, nowadays it's no big deal as exploitation frameworks are now more accessible. same applies to a lot of niche technologies that can be misused.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#283

Earlier quoted context omitted.

This trend is clearly real, but what exactly do you mean “work out how to deal with that?” One of the obvious ways to deal with it, albeit imperfectly, is in fact to keep these technologies as obscure as possible for as long as possible. Maybe an interesting norm to try to generate in this community would be, “if you plan on publishing attack X, you should publish immediately alongside it your best guess as to the ef…

Even an extreme example -- The technology to build an automated drone with a camera for targeting and a gun on a gimbal and deploy it to do a mass-shooting -- all exists today. All it would take is for some madman to pick the pieces up and assemble them. In fact, I'd be willing to bet $100 that we'll actually see some kind of tragedy to that effect in the USA within the next 5 years. The technology cats are very quic…

Your example is not extreme. Off-the-shelf "drone bombers" have already been used in the Middle East.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#284
post #281

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

[deleted]

[deleted]

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#285
post #4

Hi there, one of the maintainers here. Thanks for sharing & AMA!

Mind releasing some footage of your face and your personal information?

It would be a nice gesture to say "I am as much at risk of the consequences of my actions as the people I have chosen to put at risk"

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#286

Earlier quoted context omitted.

But what if they weren't? What if one could create something with similar destructive force with materials easily accessible, and the only hurdle is knowledge of how to do so? It's honestly an interesting moral question.

This is explored further in Nick Bostrom's Vulnerable World Hypothesis. He believes that not even a total global surveillance system would be able to stop a single teenager with the recipe for a homemade pathogen. Nothing changes about the laws of the universe in that case, only the fact that the information was not in our minds before and then it is, and we collectively cannot forget or uninvent it. We as a species…

> When Jobs held up the first smartphone and we all started cheering and fantasizing, did any of us think it would lead to this?

Jobs didn’t hold up the first smartphone and smartphone proliferation has little to do with the AI research that led to deepfakes.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#287

Earlier quoted context omitted.

That will help someone determine that a fake video didn't come from their own camera. But most videos where we're worried about deep fakes are videos that come from other people's cameras, where we don't know which signature should be valid, nor whether it should have a signature.

I believe they're saying that the manufacturer will sign the video, not the filmer. Those signatures can then be validated by platforms that the video is uploaded to. The signature isn't supposed to say "this video came from person X" it's supposed to say "this video in fact came from the real world".

But there are many manufacturers of cameras. How do I know that a particular clip should even be signed?

Signatures don’t change the provenance problem.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#288

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

Is this the first pen test tool you've seen? No, they aren't responsible, morally or legally or however you want to slice it. At all. Not one bit.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#289

Earlier quoted context omitted.

I believe they're saying that the manufacturer will sign the video, not the filmer. Those signatures can then be validated by platforms that the video is uploaded to. The signature isn't supposed to say "this video came from person X" it's supposed to say "this video in fact came from the real world".

But there are many manufacturers of cameras. How do I know that a particular clip should even be signed? Signatures don’t change the provenance problem.

It seems like the goal would be to make this ubiquitous enough that eventually you'll expect all clips to be signed, although this would of course take quite a long time to achieve, which would indeed undermine the authenticity guarantees a system like this could provide.

It seems like official channels like CSPAN, white house press briefings, and other world governments could get immediate benefit by applying these methods to all of their official communiques.

Post reply on HN