Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

131–140 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#131

Post by pbhjpbhj »

jacquesm wrote: post #93
throw9871928 wrote: post #76

I work at Axis, which makes surveillance cameras.[0] This comment is my own, and is not on behalf of the company. I'm using a throwaway account because I'd rather not be identified (and because surveillance is quite controversial here). Axis has developed a way to cryptographically sign video, using TPMs (Trusted Platform Module) built into the cameras and embedding the signature into the h.264 stream.[1] The video c…

That will just move the hack one level further down and will create even more confusion because then you'll have a 'properly signed video stream' as a kind of certificate that the video wasn't manipulated. But you don't really know that, because the sensor input itself could be computer generated and I can think off the bat of at least two ways in which I could do just that.

Isn't the point to prove physical access to the camera. As in, this stream originated at this TPM which was sold as part of this camera.

So, the best you get is that the stream shows it wasn't produced with access to a particular camera. Then impersonating a YouTuber, say, requires access to their physical camera.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#132

Post by karmakurtisaani »

belter wrote: post #125
sva_ wrote: post #114

To those who ask about the ethics of releasing something like this, I'd say that this technology already exists, and bad actors probably already can get access if they really want to and are sophisticated enough. Making this available to the general public will spread awareness of the existence of such tools, and can then possibly have a preventive effect.

You are right. I saw some guy that looked like Matt Damon trying to sell me some crypto coins...

It would be so ridiculously crazy that an established A-list actor would willingly promote those crypto scams. Must have been a deep fake!

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#133

Post by Planktonne »

ramblerman wrote: post #68
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

> If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. That's a dangerous precedent. Would you apply the same logic to a kitchen knife? Or if for some reason only freemium products count (not sure why), then a pentesting tool? I understand the underlying point you are trying to make but what are you proposing as an alternative exactly? Who gets to…

A knife is a multi-purpose tool that can be used as a weapon, and one that has so many important non-weapon uses that not having it would cause far more harm than having it would. This is intended primarily as a weapon, even when used defensively. There's an important qualitative difference there, though it's tempting to gallop down the slippery slope.

Regardless, I'm not proposing to ban either knives or this tool - I've been very clear about that. I do think that - with anything that has potential for harm - it's important for people to consider the possible consequences and to actually engage in the discussion about usage, rather than either washing their hands of the issue or declaring that any consideration will soon lead to arresting everyone for everything.

This is a path we've trodden countless times. With some things, we've collectively decided that no controls are necessary. With other things - poisons, nuclear weapons, a host of things depending on location - society has enacted controls of various efficacy and validity.

The responsible choice, regardless of whether you end up being for or any any given restriction for [thing], is to spend time thinking about it, discussing it, and - particularly when you've chosen to release something - acknowledging the potential issues.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#134

Post by FollowingTheDao »

rolisz wrote: post #94
ethanbond wrote: post #64

Earlier quoted context omitted.

Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.

For nuclear weapons, even if plans were publicly available, getting the right kind of uranium would still be a major hurdle for 99.9% of the people.

But what are the consequences of the .1% of people who get over that hurdle?

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#135

Post by IshKebab »

mustyoshi wrote: post #103

I'm glad they released this. I'm sick snd tired of seeing big companies and orgs (Google is the most recent) publish an amazing application of ML but refuse to release the trained model because the model is biased and may be used in a bad way.

I suspect that's mostly an excuse and they just want to keep it to themselves for commercial reasons. I mean I'm sure they are happy not to have to deal with any ethical issues by keeping it private but that's probably a secondary motivation.

It's not like they release their state of the art ML stuff when there aren't any ethical issues anyway, e.g. for voice recognition.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#136

Post by Hendrikto »

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

If I stab someone with a knife, who is responsible?

The inventor of knives? The knife‘s manufacturer? The store who sold me the knife? I would say the responsibility lays 100% with myself.

I do not think it makes sense to pursue inventors for what happens to their creations, unless they actively encourage misuse.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#137

Post by Planktonne »

ls15 wrote: post #55
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

Unless we cut off the chain of responsibility somewhere, the creators of the programming language, the computer and its components, as well as the people who have designed the computer and those who mined the required minerals are responsible as well.

> Unless we cut off the chain of responsibility somewhere

We can do that. We do it all the time, assigning varying levels of blame to different parties for different things.

Determining between proximal and ultimate causes, or assigning more weight to one cause or another, is not some impossible burden.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#138

Post by FollowingTheDao »

bjt2n3904 wrote: post #113
mensetmanusman wrote: post #107

Earlier quoted context omitted.

Definitely useful for scammers.

Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going…

But you are making the argument that releasing it is protective, and there is no basis for that argument.

> Society must unlearn that seeing is believing.

A total lack of trust of our shared perception? Let me know how that works out. Might as well have everyone on acid 24/7.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#139

Post by golergka »

Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

You are a partly reason for misuse, but you are not morally responsible for it. Causal relationship doesn't necessarily imply a moral one.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#140

Post by Planktonne »

bjt2n3904 wrote: post #100
Planktonne wrote: post #44

Earlier quoted context omitted.

I think that's part of what makes me uneasy about this - it's a solely legal disclaimer for something with a moral dimension, and it reads to me like the abdication of responsibility, rather than just a shield against litigation.

Oh no! We didn't remind our users to be kind to each other! We're software developers, not kindergarten teachers. How do you feel about Sqlite's code of conduct? Because if you don't like that, then you shouldn't be making moral statements in your own repositories.

Kindergarten teachers, as people who can have significant positive or negative impact, are bound by codes of ethics and practice.

This is also true of almost every other area of human effort, whether implicit or explicit. As a catch-all baseline, we have the various legal systems.

The fact that a significant proportion of the tech industry refuses point-blank to even discuss the idea of ethics is not a sign of superiority but immaturity.

Post reply on HN