Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

github.com

Post reply on HN
Print view View original

151–160 of 336 posts

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#151

Post by jacquesm »

IshKebab wrote: post #141
jacquesm wrote: post #93

Earlier quoted context omitted.

That will just move the hack one level further down and will create even more confusion because then you'll have a 'properly signed video stream' as a kind of certificate that the video wasn't manipulated. But you don't really know that, because the sensor input itself could be computer generated and I can think off the bat of at least two ways in which I could do just that.

"That will just make it more difficult to make fakes." Yes that's kind of the point. Plus I'm sure they could put the whole camera in a tamper resistant case. They could make it very difficult to access the sensor. Including focus data should make "record a screen" a bit harder too. I guess recording a projection would be pretty hard to detect, but how likely is it that people would go to those lengths, vs using a si…

> "That will just make it more difficult to make fakes."

Please don't use quotes when you are not quoting.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#152

Post by galangalalgol »

mgdlbp wrote: post #129
galangalalgol wrote: post #108

Earlier quoted context omitted.

A screen with double the resolution and twice the framrate should be indistinguishable. Moreover if you pop the case on the camera and replace the sensor with something fed by display port (probably need an fpga to convert display port to lvds, spi,ic2 or whatever those sensors use, at speed) that should work too.

Not if the camera includes metadata like focus, shutter speed, accelerometer, GPS, etc. I don't really know, but I imagine the hardware security required wouldn't be too far from what's common now. Cameras are already unrepairable, so I suppose the arguments would have to be more from the privacy and who-controls-the-chipmakers perspectives.

GPS spoofers are available legally, you just replace the gps antenna with the spoofer, so no FCC violation. You'd have to break the law if you don't want to open the case to get to the antenna. I don't have any answers to the accelerometer or focus other than replacing those sensors too, and if you made the accelerometer on the same tpm enabled soc it would make moving shots like from a dash cam hard.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#153

Post by Planktonne »

systemvoltage wrote: post #121
Planktonne wrote: post #7

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can be created, will be created. However, that doesn't free you from all moral culpability. If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I'm not saying that they shouldn't have created…

This is morally corrupt, dangerous and would lead to an oppressive, violent society. A knife maker killed for murders, a watch maker killed for the tyranny of time. We should do the exact opposite. Science and reason would cease to exist otherwise. Individuals wouldn't need to be held accountable, innovators/engineers/scientists/entrepreneurs would be. End of a free society. Have you thought of the chilling effect th…

This is not a good faith argument.

I've been extremely clear throughout this comment section - I don't want censorship, I don't think this should be banned, and nowhere have I called for legal consequences for releasing this. I want people to think about their actions, and to discuss the ethical issues arising from them.

And yet you've jumped to calling me morally corrupt because I want to murder craftsmen. That's not a reasonable reading of my comments, or in any way a proportionate response.

If you want to talk about chilling effects and the importance of science and reason, how would you describe your comments? You're shouting down discussion with wild accusations.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#154

Post by SXX »

giorgiop wrote: post #4

Hi there, one of the maintainers here. Thanks for sharing & AMA!

I just wonder. How well will this work with less realistic textures? Like imagine applying some textures on video game 3D model?

And yeah please don't be discouraged to continue publishing your work and models because some people think it's can be abused. Bad guys always have access to the tech they want anyway.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#155

Post by yosito »

jacquesm wrote: post #61
yosito wrote: post #60

Earlier quoted context omitted.

It doesn't matter what anyone thinks about it.

I actually disagree with that. I think software and ethics are closely intertwined and too often we see them as disjoint. It's interesting to try to find a software project that does not have an ethically dubious angle to it, that isn't all that simple.

My point is not that ethics don't matter, but rather that accessible deep fake technology has been on the verge of existing for years already. It was only a matter of time. You can have any ethical opinion about it that you want. But reality is that unverified video is no longer a trustworthy medium. If anything this software release makes that fact harder to ignore. We will have to adjust sooner rather than later.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#156

Post by pixl97 »

Planktonne wrote: post #126
bjt2n3904 wrote: post #113

Earlier quoted context omitted.

Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going…

I don't think anyone so far has called for this to be unpublished, just for it to be more carefully considered and for the ethical aspect to be addressed when publishing. That's part of 'building a culture of distrust'. Knee-jerk refusal to even consider thinking/talking about the use of technology is far more irresponsible than any other stance.

Ignoring human nature is far worse. There are people that will release this simply because you tell them no.

Even beyond that, if you're an authoritarian nation with a rather good lockdown on your news and internet, what's the downside of throwing tools like this at democracies?

In my view the age of tools like this has already been here for some time and they are going to their next step of evolution, and there is very little we can do about preventing the distribution of them without affecting other pieces of software and their distribution.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#157

Post by jacquesm »

yosito wrote: post #155
jacquesm wrote: post #61

Earlier quoted context omitted.

I actually disagree with that. I think software and ethics are closely intertwined and too often we see them as disjoint. It's interesting to try to find a software project that does not have an ethically dubious angle to it, that isn't all that simple.

My point is not that ethics don't matter, but rather that accessible deep fake technology has been on the verge of existing for years already. It was only a matter of time. You can have any ethical opinion about it that you want. But reality is that unverified video is no longer a trustworthy medium. If anything this software release makes that fact harder to ignore. We will have to adjust sooner rather than later.

> It was only a matter of time.

This is a cop out. Yes, it is a matter of time, but still it takes people to build this stuff and they are rarely the people that use/abuse it.

> But reality is that unverified video is no longer a trustworthy medium.

No, video is no longer a trustworthy medium and in many ways never was.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#158

Post by Diggsey »

throw9871928 wrote: post #83
Tepix wrote: post #80

Earlier quoted context omitted.

> it will pretty much solve the trust issues I'm not so sure. How will you verify a signature if you see a video on TV or on social media? Do you believe these devices are 100% secure and the keys will never be extracted?

Nothing is 100% secure, but the point of using a TPM is to prevent extraction of the keys.

TPMs are great at preventing extraction of keys via the exploitation of software bugs. They are not so good at preventing extraction if you have physical access to the machine, and especially not if you don't need the machine to survive intact, and can afford to destroy as many machines as you need to get the keys out...

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#159

Post by systemvoltage »

Planktonne wrote: post #153
systemvoltage wrote: post #121

Earlier quoted context omitted.

This is morally corrupt, dangerous and would lead to an oppressive, violent society. A knife maker killed for murders, a watch maker killed for the tyranny of time. We should do the exact opposite. Science and reason would cease to exist otherwise. Individuals wouldn't need to be held accountable, innovators/engineers/scientists/entrepreneurs would be. End of a free society. Have you thought of the chilling effect th…

This is not a good faith argument. I've been extremely clear throughout this comment section - I don't want censorship, I don't think this should be banned, and nowhere have I called for legal consequences for releasing this. I want people to think about their actions, and to discuss the ethical issues arising from them. And yet you've jumped to calling me morally corrupt because I want to murder craftsmen. That's no…

I am discussing larger societal implications of what happens when this type of morality expands to the extreme.

Absolutely zero to do with you personally. You're building a strawman.

To you personally: If you're going around touting "other side's morality", you should be open to seeing what happens when this kind of thinking is adopted by the society. Don't play the victim game.

Re: Deepfake Offensive Toolkit (real-time deepfakes for virtual cameras)

#160

Post by nonrandomstring »

pid-1 wrote: post #122
nonrandomstring wrote: post #20

Earlier quoted context omitted.

In cybersecurity the bad guys always win. There are two very rough "proofs of a kind" I use to explain this to students, both well known in military and counter-terrorism analysis. One is called Blotto analysis. As the number of fronts grows large (>12 is an important point) the game favours the attackers. Defenders have to always win on all fronts, but terrorists only have to win once on any of n fronts. Another sce…

> Defenders have to always win on all fronts, but terrorists only have to win once on any of n fronts It depends on how you design systems and processes. Let Mn denote the nth defense mechanism of a system. We can build architectures in which the probability of breaches is calculated as: (M0 is pwned) AND (M1 pwned) AND (M2 is pwned) AND (M3 is pwned)..... You can see the total probability can become small quickly, a…

Yes, I haven't mentioned defence in depth here. And as you say, serious real world applications are more than a single layer. But of course what we see with DiD is polynomial growth of complexity, so policy, monitoring, rebuild time and so much else gets hard to manage - upshot is someone, somewhere falls back to simpler configurations - and there's the entry point. But sure, thank goodness we've moved beyond a single blacklist firewall these days :)
Post reply on HN