Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

191–200 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#191

Post by trasz »

FollowingTheDao wrote: post #138
bjt2n3904 wrote: post #113

Earlier quoted context omitted.

Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going…

But you are making the argument that releasing it is protective, and there is no basis for that argument. > Society must unlearn that seeing is believing. A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7.

>A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7.

It's no different from religion, which was the backbone of the western civilisation for quite some time.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#192

Post by HWR_14 »

blagie wrote: post #150
sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

As someone with a stalker, I can't emphasize this enough. A stalker will go to all sorts of lengths to do bizarre shit. People don't believe it. I would guess governments will do some equivalent there-of. Democratizing access to things -- including bad things -- has a preventative effect: 1) I can guard against things I know about 2) People take me seriously if something has been democratized The worst-case scenario…

Your "worst case" depends greatly on who the select group is. Is it movie studios making 9 figure budgets, or is it any obsessed person who can figure out how to find and install software.

Obviously, it's hard to imagine many situations like that, but you can imagine a process that required a 8-figure quantum supercomputer.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#193

Post by HWR_14 »

belter wrote: post #125
sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

You are right. I saw some guy that looked like Matt Damon trying to sell me some crypto coins...

For what it's worth, actors seem to be securing themselves against this using IP rights.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#194

Post by mensetmanusman »

IshKebab wrote: post #144
mensetmanusman wrote: post #112

This is like releasing an app that unlocks all car doors and telling users to only use it on their car.

You can buy lock picks specifically for cars. The world hasn't ended.

All cars…

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#195

Post by mensetmanusman »

trasz wrote: post #191
FollowingTheDao wrote: post #138

Earlier quoted context omitted.

But you are making the argument that releasing it is protective, and there is no basis for that argument. > Society must unlearn that seeing is believing. A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7.

>A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7. It's no different from religion, which was the backbone of the western civilisation for quite some time.

Religion is a contemplation of free will, not sure if contemplating that is like being on acid :)

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#196

Post by wussboy »

Karliss wrote: post #187
sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

Even without deepfakes any kind of system relying on a person (or computer) not being tricked by webcam video seems quite questionable. People could still be tricked with a spliced video fragments of the real person or makeup especially if the set of face expressions used during "liveness check" is known ahead of time.

I try to imagine how society will deal with this. What if deep fakes are so perfect that anyone can generate real-time footage of anyone else doing anything? As a society we’d need to move out of the virtual and back into the physical. Would that be such a bad thing?

I suspect this perfect deep fake technology might be a real boon to society.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#197

Post by npteljes »

throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

>it will pretty much solve the trust issues deep fakes are causing.

It's a nice piece of tech, I can see it being used in court for example, to strengthen the claim that a video is not a deepfake.

However, that's not "the" problem with deepfakes. Propaganda of all sorts have demonstrated that "Falsehood flies, and the Truth comes limping after it". As in, with the proper deepfakes, you can do massive damage via social media for example. People re-share without validation all the time, and the existence of deepfakes add fuel to this fire. And I think that we can't do anything about either.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#198

Post by thereddaikon »

galangalalgol wrote: post #152
mgdlbp wrote: post #129

Earlier quoted context omitted.

Not if the camera includes metadata like focus, shutter speed, accelerometer, GPS, etc. I don't really know, but I imagine the hardware security required wouldn't be too far from what's common now. Cameras are already unrepairable, so I suppose the arguments would have to be more from the privacy and who-controls-the-chipmakers perspectives.

GPS spoofers are available legally, you just replace the gps antenna with the spoofer, so no FCC violation. You'd have to break the law if you don't want to open the case to get to the antenna. I don't have any answers to the accelerometer or focus other than replacing those sensors too, and if you made the accelerometer on the same tpm enabled soc it would make moving shots like from a dash cam hard.

Its not like TPMs are infallible. And even if they are thought secure today, older encryption becomes trivial to crack with time. But like you said, its about raising the bar. You can do a lot to mitigate the threat of deepfakes to a certain point which will eventually push them back into just the realm of those who really know what they are doing. That's not ideal, but well funded and talents groups have been able to falsify evidence to discredit people since the beginning of time. So the nature of the problem doesn't change, they just have another tool.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#199

Post by jumpkick »

Hendrikto wrote: post #136
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

If I stab someone with a knife, who is responsible? The inventor of knives? The knife‘s manufacturer? The store who sold me the knife? I would say the responsibility lays 100% with myself. I do not think it makes sense to pursue inventors for what happens to their creations, unless they actively encourage misuse.

Parents of one of the victims of the recent elementary school shooting in Uvalde, Texas are attempting, or preparing, to sue the gun manufacturer of the semiautomatic rifle that was used in that event. Will they sue, will they win? I don't know.

Do guns kill people or do people kill people? Could be a relevant analogy.

https://www.npr.org/2022/06/03/1102755195/uvalde-special-ed-...

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#200

Post by light_hue_1 »

sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

That's like saying that nuclear weapons exist, and bad actors can potentially get them, so let's lower the bar so that anyone can.

Making such tools accessible is reprehensible. It will lead to more bad actors, to less trust in media and in any objective reality, and more erosion of our institutions and society.

There is absolutely no reason whatsoever for this. It's unethical and frankly downright evil.

Post reply on HN