Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

271–280 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#271

Post by rockemsockem »

kube-system wrote: post #251
throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

That will help someone determine that a fake video didn't come from their own camera. But most videos where we're worried about deep fakes are videos that come from other people's cameras, where we don't know which signature should be valid, nor whether it should have a signature.

I believe they're saying that the manufacturer will sign the video, not the filmer. Those signatures can then be validated by platforms that the video is uploaded to. The signature isn't supposed to say "this video came from person X" it's supposed to say "this video in fact came from the real world".

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#272

Post by rockemsockem »

driverdan wrote: post #171
throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

I don't see how that would be useful here. I'm not giving out info about my webcam to anyone. There would be no way to verify the signature.

Manufacturer signature, not your signature.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#273

Post by rockemsockem »

throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

That's really cool! I've been waiting for tech like this to finally come to light. Honestly expected either Google or Apple to lead the way on it. Have you all worked with the content authenticity initiative at all? It seems like they're looking at ways to develop standards around tech like this to ensure interoperability in the future.

https://contentauthenticity.org/

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#274

Post by rockemsockem »

ethanbond wrote: post #269
rockemsockem wrote: post #264

Earlier quoted context omitted.

I'd argue that one of the biggest reasons deepfakes pose a threat is because a large number of people don't know they exist. It's just those e-mail chains from the early 00s with poorly photoshopped images that lots of people thought were real. Now that the public knows broadly that digital manipulation is easy and they have seen it with their own eyes a ton things like that are believed less. Notice I do say less th…

Right but this means real content is less believable, too.

Indeed. Honestly that sounds like a bit of a plus too. A bit more skepticism should be applied to videos of talking heads in suits saying things. Especially when there's a huge panel of them.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#275

Post by Shared404 »

rockemsockem wrote: post #264
phkahler wrote: post #254

Earlier quoted context omitted.

>> Does this apply to nuclear weapons? To biological weapons? That may not be a fair comparison. Nuclear weapons are difficult to make, and using them will not go undetected - the damage would be enormous. Deep fakes are a real threat today, and anyone willing to pay for the expertise (which isn't that hard to find) can have them made. By putting this out there, deepfakes are now something almost anyone can make whic…

I'd argue that one of the biggest reasons deepfakes pose a threat is because a large number of people don't know they exist. It's just those e-mail chains from the early 00s with poorly photoshopped images that lots of people thought were real. Now that the public knows broadly that digital manipulation is easy and they have seen it with their own eyes a ton things like that are believed less. Notice I do say less th…

I would agree with this.

It may be worthwhile to have someone create a bunch of deepfakes of politicians saying "This video has been faked" to drive awareness of this as a threat model.

E: Forgot the key bit, it would need to be spammed all over Facebook/Instagram/Mainstream social media.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#276

Post by bergenty »

barnabee wrote: post #43
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

No post body was provided.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#277

Post by polartx »

ospray wrote: post #253
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

The main value in releasing tools like this is to demonstrate weakness in our current security controls. A key weakness of biometrics is that there is no secret data. Open source tooling like this help people understand that.

I wish I didn’t have to scroll this far to find your rational perspective. Lets take the famous LockPickingLawyer of YouTube, is he responsible for every crime where the thief defeats a lock that he has demonstrated the weaknesses of? I would say “no!”. Exposing a weakness puts the onus of securing said weaknesses on those that sell technology/devices/services that market themselves as “secure”.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#278

Post by nonbirithm »

ethanbond wrote: post #269
rockemsockem wrote: post #264

Earlier quoted context omitted.

I'd argue that one of the biggest reasons deepfakes pose a threat is because a large number of people don't know they exist. It's just those e-mail chains from the early 00s with poorly photoshopped images that lots of people thought were real. Now that the public knows broadly that digital manipulation is easy and they have seen it with their own eyes a ton things like that are believed less. Notice I do say less th…

Right but this means real content is less believable, too.

I think a lot of information could be treated like pollution, in that it is easily created, spreads quickly, and becomes harder to clean up the more people are able to learn of it. I have to wonder if forging on like this will have a net positive effect on society, or if these kinds of developments are just fated by human nature.

In a perfect world, what should have happened was a separate team inventing a reasonably accurate deepfake detector to the world before the deepfake producer gained prominence (differential technological development), but interest, productivity and excitement on the researchers' part to release their work first seems to have foiled that.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#279

Post by momojo »

madrox wrote: post #207

The moral lens people apply to deepfakes feels myopic to me. There are lots of tools that have been built that have arguably done more harm that no one talks about, like port scanning tools. Perhaps because the the negative consequences are so visually obvious and require few intuitive leaps.

> Perhaps because the the negative consequences are so visually obvious and require few intuitive leaps.

Exactly this. At some point it's not feasible to calculate the impact of a technology's trickle down. Unless the impact is large enough.

I'm not surprised people have such a viscerally moral reaction to deepfakes.

> There are lots of tools that have been built that have arguably done more harm that no one talks about, like port scanning tools

I don't think this is a fair argument against current tools.

If we had some way of accurately measuring the secondary effects of port-scanners, should we start caring? Should we retroactively remove any tools that we later regret? (I don't have good answers to these either)

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#280

Post by jacquesm »

teakettle42 wrote: post #237
jacquesm wrote: post #41

Earlier quoted context omitted.

Yes, they did that because they did what everybody in that position would do: pick the argument that is most likely to find sympathy with a jury. Also: note that the degree to which everything is labeled 'speech' is fairly uniquely American.

This argument echos the same justifications made for the US’ cryptography embargoes in the 1990s. (Rightfully) labeling code as speech is what allowed us to break down that barrier. Would you also be questioning the ethics of djb in 1995? https://en.wikipedia.org/wiki/Daniel_J._Bernstein#Bernstein_...

No, it doesn't.

Because once created something will have a life of its own. The question is only meaningful prior to creating it. This goes for the atomic bomb, the invention of gunpowder, strong cryptography and any other tool.

But if you don't care about ethics then it is obvious why you would not ask that question to begin with and I think that technology without ethics is no better than your average arms manufacturer. Only there at least we have - in some cases - some control over the uses, whereas software multiplies at a speed that politics can not catch up with. This may be a good thing, or it may be a terrible thing, and if it is the latter you will only find out when it is too late if you don't spend some cycles thinking about it beforehand.

I liked the internet a lot better before I had to think about the security implications of every move I made, and the difference between 'bad actors' that are incapable of wrecking everything and 'bad actors' that are empowered by the brain children of the good actors far more than the good actors are (due to the asymmetry between destruction and creation) may be more than we can deal with.

Post reply on HN